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FOREWORD 


Several  years  ago,  the  US  Army  CECOM  Center  for  Night  Vision 
and  Electro-Optics  (C2NVE0)  was  faced  with  the  problem  of  evaluat¬ 
ing  the  performance  of  Automatic  Target  Recognizers  (ATR)  before 
going  out  to  conduct  expensive  field  tests.  We  used  a  TV  camera  to 
simulate  infrared  inputs  on  a  physical  terrain  board  so  that  we 
could  control  conditions  and  thus  have  a  known,  repeatable  input  to 
enable  a  statistically  valid  test.  However,  there  are  some  draw¬ 
backs  to  a  physical  terrain  board,  such  as  a  lack  of  dynamics, 
limited  number  of  scenes,  single  spectrum,  single  sensor,  etc.  In 
order  to  advance  the  evaluation  technology  and  include  human  per¬ 
formance  as  well  as  multisensor  processors  and  algorithms,  the  use 
of  a  digital/electronic  terrain  board  (D/ETB)  is  required. 

A  D/ETB  is  conceptually  analogous  to  a  physical  terrain  board 
in  that  it  is  expected  to  be  a  computer  resident,  three-  or  more 
dimensional  simulation  of  the  real  world  that  simultaneously  repre¬ 
sents  the  scene  in  all  useful  spectrums  including  acoustic,  radar, 
laser  radar,  thermal,  visible,  etc.  Additionally,  the  D/ETB  should 
be  capable  of  simulating  various  atmospheric  and  active  sensor 
effects,  as  well  as  moving  targets  and  sensor  platforms. 
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The  Digital/Electronic  Terrain  Board  Symposium  was  organized 
for  the  objectives  of  providing  an  opportunity  for  those  working  in 
the  field  to  discuss  and  demonstrate  their  state-of-the-art  to 
peers;  and  to  uncover  areas  of  duplication  and,  more  importantly, 
gaps  in  these  efforts.  I  believe  that  both  objectives  were  met, 
thanks  to  the  excellent  work  provided  by  the  authors  and  speakers. 
These  Proceedings  are  a  permanent  record  of  their  work,  which  will 
allow  others  in  the  industry  to  share  in  their  knowledge  and  under¬ 
standing,  and  lead  to  an  operational  Digital/Electronic  Terrain 
Board  in  the  future. 

Every  successful  endeavor  depends  upon  the  support  and  efforts 
provided  by  many  people,  so  I  would  like  to  thank  the  following  for 
assistance  in  helping  us  conduct  the  first  Digital/Electronic 
Terrain  Board  Symposium. 

Boeing  Military  Airplane  Company,  for  use  of  their  facilities 
to  hold  the  Symposium;  and  especially  Dr.  Shelly  Eucker  for  hosting 
the  co-sponsored  ATRWG  Meeting  and  Symposium. 

The  Automatic  Target  Recognition  Working  Group  (ATRWG) ,  par¬ 
ticularly  Lloyd  Clark  and  Bonita  Moutoux,  for  organizing  the  co¬ 
sponsored  ATRWG  Meeting  and  Symposium. 

The  I IT  Research  Institute,  and  Andrew  Zembower,  for  organiz¬ 
ing  publication  of  the  Symposium  Proceedings. 


vi 


The  speakers  who  presented  papers  at  the  Symposium:  Scott 

Brandt,  Honeywell;  William  Cameron,  Boeing  Aerospace  &  Electronics; 
Syama  Chaudhuri,  Sensor  Data  Integration  Inc.;  Geoffrey  Gardner, 
Grumman  Data  Systems;  Jeffrey  Gillberg,  Honeywell;  Laslo  Greczy,  US 
Army  Environmental  Topographic  Laboratory;  Conrad  Hoeppner,  AAI 
Corporation;  Michael  Lander,  US  Army  CECOM  Center  for  Night  Vision 
&  Electro-Optics;  Jerry  Lent,  Image  Data  Corporation;  Charles 
Lindahl,  Texas  Instruments;  Maximo  Lorenzo,  US  Army  CECOM  Center 
for  Night  Vision  &  Electro-Optics;  Dennis  Lui,  US  Army  CECOM  Center 
for  Electronic  Warfare,  Reconnaissance,  Surveillance  and  Target 
Acquisition;  Jay  Mendelsohn,  Grumman  Corporate  Research  Center; 
Richard  Ramroth,  Northrop  Electronics  System  Division;  Dale 
Robison,  US  Naval  Weapons  Center;  Timothy  Rogne,  OptiMetrics  Inc.; 
Richard  Shirkey,  US  Army  Atmospheric  Sciences  Laboratory;  Lori 
Scarlatos,  Grumman  Data  Systems;  and  Craig  Yandow,  Hughes  Aircraft 
Company . 

The  Symposium  guest  speakers:  Dr.  Kimball  Kramer,  Defense 
Advanced  Research  Project  Agency;  and  Dr.  Rudolf  G.  Buser, 
Director,  US  Army  CECOM  Center  for  Night  Vision  and  Electro-Optics. 

Most  importantly,  to  Jill  Bloxam  for  her  assistance  and 
untiring  efforts  in  organizing  the  Symposium. 

CARL  W.  HOOVER,  JR. 

CHIEF,  LABORATORY  EVALUATION 

VISIONICS  DIVISION,  C2NVE0 
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PROBLEMS 


•  REAL-TIME  OPERATION 

•  HIGH  DEVELOPMENT  COSTS 

•  MODERATE  INITIAL  COSTS 

•  PERCEPTION  OF  LACK  OF  FIDELITY 

•  VERIFICATION  (NON- VISIBLE  SPECTRUM) 

•  FIDELITY  THRU  FEEDBACK 

•  PERCEPTION  OF  THE  IMPOSSIBILITY  OF 

REPLACING  CAPTIVE  FLIGHT  TESTS 

•  ALL  vs.  1st  n 


DARPA-TTO  ■  KrrbaJ  Knanar  -  604-7996 
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1 


®8gfl8gi(l/il]®(gliir®iri]fl©  T©[nr§ta  l®@o^I 


( 


ADVANTAGES  (1) 


•  VERY  LOW  CONTINUING  COSTS 
COMPARED  TO  THE  COSTS  OF 
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SCENE 

METRICS  PROBLEM  (2) 


INCREASING 

ADVERSE 

WEATHER 


W.  Texas 

INCREASING  CLUTTER 
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SCENE 

METRICS  PROBLEM  (3) 


1  W.  Texas 

V  INCREASING  CLUTTER 
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SCENE 

METRICS  PROBLEM  (4) 


INCREASING 
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MEASURES 
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ADVERSE 

WEATHER 


W.  Texas 


INCREASING  CLUTTER 
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PERFORMANCE  SPECIFICATIONS 

ajk  «J» 

WE  NEED: 

A  MEASURE  OF  THE  CURRENT  STATE-OF-THE-ART 
A  MEASURE  OF  WHEN  TO  FIELD 
THE  RELATION  BETWEEN  THE  TWO 
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HOWEVER 

We  must  not  1st  PERCEIVED  INADEQUACIES  of 
systsms  In  dsvslopmsnt  and  ANTICIPATED 
IMPROVEMENTS  In  these  systems  keep  us  from 
fielding  working  systems  which  are  significantly 
better  than  what  we  have  in  the  field  now. 
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IT  MUST  ACCOMMODATE  MULTI-SENSOR  SIMULATIONS 

NEW  SENSORS  AND  NEW  SENSOR  MODES  AND  PARAMETERS 
ARE  CONTINUALLY  APPEARING 

THERE  MUST  BE  A  FLEXIBLE  "LAYERED"  ARCHITECTURE 
TO  ACCOMMODATE  THESE  AUGMENTATIONS  WITH 
MINIMUM  RE-WRITING  OF  SOFTWARE 


DARPA-TTO  •  Kimball  Kiamar  •  894-7996 


9/28/89-16 


7 


TcmriT^Otra  li@aing] 


LONG-RANGE  GOAL 


A  DoD  STANDARD 

DIGITAL/ELECTRONIC  TERRAIN  BOARD 
TEST  SET 

for  EACH  ALGORITHM  TYPE 


DARPA-TTO  ■  Kimball  K/amaf  -  604-7996 


9/28/89-1 7 


8 


Presented  at  the 

DIGITAL/ELECTRONIC  TERRAIN  BOARD  SYMPOSIUM 
5-6  October  1989  GACIAC  PR  89-07 


C2NVKO  Digital  Terrain  Board  (DTB)  Concept 

Maximo  Lorenzo,  Sandor  Der 

US  Army  CECOM  Center  for  Night  Vision  &  Electro-Optics 

Attn:  AMSEL-RD-NV-V 

Fort  Belvoir,  Virginia  22060-5677 

David  Walthall,  J.  R.  Moulton,  Jr. 

E/OIR  Measurements  Inc. 

P.  0.  Box  3348  College  Station 
Fredericksburg,  Virginia  22402 

ABSTRACT 

The  US  Army  CECOM  Center  for  Night  Vision  and  Electro-Optics 
(C2NVEO)  has  recognized  the  need  for  computer  generated  synthetic 
sensor  output  for  the  testing  and  development  of  automatic  target 
recognizers/cuers  and  for  human  observer  testing.  This  capability 
should  include  simulations  of  real  world  terrain,  vegetation, 
atmospherics  and  objects  as  well  as  simulations  of  various  sensors, 
including  FLIR,  TV,  laser  radar,  millimeter  wave  radar,  and  vibra¬ 
tion  sensors. 

1 .  INTRODUCTION 

Current  testing  and  development  of  automatic  target  recogniz¬ 
ers  (ATR)  and  cuers  rely  on  data  generated  from  field  testing  and 
from  physical  terrain  boards.  Field  data  collection  efforts  imple¬ 
ment  the  use  of  actual  sensors  on  actual  terrain  with  actual  tar¬ 
gets,  which  ensure  validity  of  the  data  used,  however  the  data 
suffers  from  several  disadvantages.  Terrain  variety  is  limited  to 
the  immediate  location  of  the  test,  which  is  usually  a  small  por¬ 
tion  of  a  military  base.  Target  variety  is  limited  to  whatever 
targets  are  on  hand  and  available,  which  often  means  that  important 
threat  vehicles  are  absent.  The  lack  of  environmental  control  over 
field  data  puts  severe  constraints  on  the  data  collected.  ATR 
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problems  that  could  be  discovered  in  a  controlled  environment  often 
are  masked  by  the  unpredictability  of  the  environment.  For  exam¬ 
ple/  if  a  range  sensitivity  test  was  conducted  by  recording  imagery 
with  targets  at  set  intervals,  it  is  undeterminable  whether  the 
changes  in  ATR  performance  were  caused  by  changes  in  range,  target 
temperature,  atmospheric  turbulence,  or  some  other  factor.  Also, 
field  testing  is  very  expensive,  which  means  that  only  a  very 
limited  set  of  data  is  collected. 

The  C2NVE0  physical  terrain  board  uses  a  TV  camera  as  the 
sensor.  FLIR  imagery  is  simulated  by  inverting  the  TV  image.  The 
targets  placed  on  the  board  are  painted  various  shades  of  gray  to 
simulate  their  temperature;  the  hotter  the  target,  the  darker  the 
paint.  Thus,  when  the  image  is  inverted,  the  target  appears  white 
hot.  The  physical  terrain  board  allows  great  control  over  target 
signatures  and  environmental  conditions,  and  is  much  less  expensive 
than  field  testing;  but  it  can  serve  only  as  a  first  step  toward 
evaluating  an  ATR.  A  physical  terrain  board  test  allows  the  ATR 
contractor  to  debug  hardware  and  software  problems  before  a  field 
test,  and  gives  evaluators  some  measure  of  contrast  required  for 
detection,  and  a  measure  of  false  alarm  rate.  The  physical  ter¬ 
rain  board  cannot  simulate  laser  radar,  millimeter  wave  radar,  or 
vibration  sensors;  the  only  target  signatures  it  can  use  are  those 
that  are  painted  onto  a  model,  which  is  difficult,  time  consuming, 
and  limits  the  variety  of  available  target  signatures-  The  physi¬ 
cal  terrain  board  has  a  limited  number  of  available  backgrounds. 
Creating  new  backgrounds,  new  vegetation  types,  or  new 
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environmental  conditions  is  expensive  because  all  work  must  be  done 
manually.  For  these  reasons,  the  physical  terrain  board  serves 
better  as  a  stationary  benchmark  than  as  a  robust  model. 

A  Digital  Terrain  Board  (DTB)  would  have  the  advantages  of  low 
cost  data  collection  and  environmental  control,  and  would  have  the 
capability  to  simulate  a  much  wider  variety  of  environmental  condi¬ 
tions  than  the  physical  terrain  board.  DTB  imagery  would  simulate 
sensor  observables  and  noise  directly,  presumably  resulting  in  more 
realistic  imagery  with  wider  usage  than  the  physical  terrain  board 
imagery.  Images  could  be  generated  of  any  terrain  for  which  a 
digital  map  exists,  and  any  targets  for  which  models  exist  could  be 
included  at  will.  The  low  cost  would  allow  a  wide  variety  of  data 
to  be  collected,  and  its  flexibility  would  allow  new  data  to  be 
collected  as  needs  arise.  This  data  could  be  used  for  ATR  algo¬ 
rithm  development  as  well  as  testing.  Currently,  ATR  algorithm 
development  is  hindered  by  a  lack  of  training  data.  It  is  not 
anticipated  that  the  DTB  would  completely  replace  field  or  physical 
terrain  board  testing;  both  forms  of  testing  still  are  needed  as  a 
control  and  to  test  the  DTB  itself.  The  issue  of  image  validation 
is  discussed  in  more  detail  at  the  end  of  this  paper. 

Current  ATR  algorithm  development  efforts  suffer  from  a  lack 
of  data  for  the  training  of  algorithms.  The  data  sets  provided 
usually  cover  a  very  limited  set  of  environmental  conditions  and 
targets.  Due  to  the  low  cost  and  ease  with  which  DTB  imagery  could 
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be  generated/  an  algorithm  developer  could  be  provided  with  a  rich 
set  of  data,  and  easily  could  request  additional  data  as  needs 
arise . 

The  DTB,  envisioned  by  C2NVE0,  is  driven  by  different  require¬ 
ments  than  most  existing  DTBs  which  are  used  for  mission  planning 
purposes.  Mission  planning  requires  real  time  or  near  real  time 
generation  of  images  which  only  serve  to  give  humans  some  idea  of 
the  appearance  of  visible  features  in  a  scene.  Real  time  genera¬ 
tion  is  unnecessary  for  our  data  collection,  but  the  images  gener¬ 
ated  must  be  adequate  for  the  testing  of  ATRs  as  well  as  humans. 
ATRs  use  various  image  metrics  to  arrive  at  a  target  recognition 
decision,  and  it  is  important  that  the  synthetic  images  have  met¬ 
rics  similar  to  the  real  world.  Different  ATRs  use  very  different 
metrics,  and  the  metrics  that  future  ATRs  will  use  cannot  be  pre¬ 
dicted;  thus,  the  authors  believe  that  the  best  approach  is  to 
model  real  world  observables  directly,  rather  than  attempt  to  force 
specific  image  metrics  in  the  synthetic  images  to  be  the  similar  to 
real  world  values.  It  is  believed  that  adequate  modeling  will 
provide  realistic  image  metric  values  for  currently  used  metrics  as 
well  as  future  ones,  while  forcing  of  specific  metrics  may  cause 
the  DTB  to  be  useless  for  the  development  and  testing  of  ATRs  which 
use  other  metrics. 

Abundant  computer  facilities  and  relaxation  of  the  real  time 
requirement  will  allow  the  use  of  computationally  intensive  algo¬ 
rithms.  This  will  allow  us  to  concentrate  on  using  the  most 
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realistic  algorithms  available,  while  temporarily  overlooking  time 
constraints.  Attention  will  be  given  to  speeding  up  the  algorithms 
only  after  a  given  synthetic  scene  generator  is  performing  accept¬ 
ably  . 


One  of  the  biggest  problems  common  to  field  and  physical 
terrain  board  data  collection  is  the . lack  of  flexibility  in  choos¬ 
ing  the  data.  Consequently,  a  great  deal  of  attention  will  be 
given  to  maintaining  flexibility  in  the  design  of  the  DTB,  which, 
along  with  fidelity,  are  the  driving  requirements  behind  this 
project . 

The  synthetic  scene  generators  will  take  input  from  a  common 
set  of  files  to  ensure  uniformity  and  reduce  duplication  of  effort. 
These  consist  of  dynamic  files  which  are  specified  by  the  event 
control  module,  and  stationary  files  which  have  data  that  is  fixed 
for  every  scenario.  The  dynamic  files  contain  the  vehicle  activi¬ 
ty,  sensor  activity,  and  meteorological  activity  file.  The  sta¬ 
tionary  files  contain  the  vehicle,  object,  vegetation,  and  back¬ 
ground  data  files.  Each  of  these  files  are  described  in  more 
detail  below.  It  should  be  noted  that  these  files  are  subject  to 
change  as  work  on  the  DTB  progresses. 

2 .  The  Event  Control  Module 

The  event  control  module  will  serve  as  the  interface  between 
the  user  and  the  entire  DTB  system.  The  user  will  use  this 
interface  to  completely  define  the  scenario.  Thus,  all  dynamic 
input  to  che  DTB  system  will  be  chosen  in  this  module.  Dynamic 
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input  is  all  information  that  is  specific  to  a  given  scenario,  such 
as  target  sets,  flight  paths,  real  world  location  of  vehicles  and 
sensors,  and  weather  conditions.  This  information  is  contained  in 
the  vehicle  activity,  meteorological  activity,  and  sensor  activity 
files,  which  are  described  in  more  detail  below.  It  is  envisioned 
that  the  user  will  be  able  to  create  new  files  or  use  existing 
ones . 


3 .  Vehicle  Activity  File 

The  vehicle  activity  file  will  specify  all  the  activities  of 
all  vehicles  included  in  the  scenario  which  are  relevant  to  any  of 
the  sensors  being  simulated.  A  few  of  these  are  stated  here,  but 
it  is  not  intended  that  this  list  be  all  inclusive.  It  is  expected 
that  as  the  synthetic  scene  generation  programs  are  being  de¬ 
veloped,  the  relevance  of  other  actions  to  a  targets'  signature 
will  be  realized,  and  these  actions  will  be  included  in  the  file. 
It  should  be  kept  in  mind  that  while  not  all  of  these  will  be 
implemented  from  the  outset,  including  them  here  will  ensure  that 
they  are  not  forgotten. 

The  first  group  of  activities  in  the  file  actually  will  be  a 
history  of  operations  before  the  scenario  started,  including  those 
activities  that  occur  at  a  time  t<0  that  effect  the  signature  at 
t=0 .  Examples  would  include:  firing  of  the  main  gun,  running  of 
the  engine,  movement  of  the  vehicle,  and  miscellaneous  items  such 
as  running  the  heater.  The  effect  of  the  first  three  on  the 
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thermal  signature  of  the  vehicle  is  obvious,  and  the  fourth  may 
have  relevance.  It  would  be  incumbent  upon  the  designers  of  the 
synthetic  thermal  signature  generator  to  make  that  determination. 

The  file  also  needs  to  include  the  activities  of  the  vehicle 
during  the  scenario  being  run  which  are  relevant  to  the  signature 
for  any  sensor.  These  activities  are  dynamic  in  the  sense  that  it 
is  necessary  to  know  them  through  the  duration  of  the  scenario, 
thus  all  of  these  activities  will  be  expressed  as  a  function  of 
time.  This  would  include  vehicle  location  on  the  terrain  board, 
vehicle  facing,  the  facing  of  the  turret  if  applicable,  whether  the 
engine  is  running,  whether  the  weapons  on  the  vehicle  are  being 
fired,  and  whether  a  stationary  vehicle  is  dug  in,  in  defilade,  or 
using  camouflage.  These,  too,  are  considered  dynamic  because  they 
could  change  within  the  time  period  of  a  scenario. 

4 .  Sensor  Activity  File 

The  sensor  activity  file  will  specify  the  location,  attitude, 
and  settings  of  all  the  relevant  sensors  for  each  time  period  in 
the  duration  of  the  scenario,  and  will  specify  the  sensor  platform 
in  use.  This  would  direct  the  synthetic  scene  generator  to  a 
platform  file,  which  would  specify  the  vibration  inherent  in  the 
platform,  as  well  as  information  on  the  gimbal,  such  as  gimbal 
vibration  and  speed. 
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5.  Meteorological  Data  File 


This  file  will  include  relevant  weather  information  during  the 
scenario  as  well  as  for  some  as  yet  unspecified  time  before  the 
beginning  of  the  scenario.  Meteorological  effects  on  signatures 
are  not  well  understood,  so  it  is  quite  likely  that  the  factors 
specified  here  will  prove  to  be  incomplete.  The  factors  listed  and 
the  rationale  behind  their  inclusion  may  serve  as  a  guide  to  the 
thoughts  of  the  authors  on  the  complexity  of  the  DTB  problem. 

Solar  loading  both  during  and  prior  to  the  scenario  is  one  of 
the  more  obvious  meteorological  effects  on  object  and  background 
observables  for  the  TV  and  FLIR  sensors.  The  history  of  solar 
loading  is  particularly  essential  for  estimating  infrared  observa¬ 
bles.  Wind  information  is  necessary  to  estimate  object  and  back¬ 
ground  temperature,  to  simulate  the  movement  of  wind  movable  ob¬ 
jects,  and  to  account  for  atmospheric  turbulence  effects  on  laser 
radar  (Ladar)  output.  Precipitation  and  humidity  alter  the  atmos¬ 
pheric  transmission  of  both  infrared  and  light  waves.  Clouds 
selectively  mitigate  the  effects  of  solar  loading,  and  often  will 
appear  in  the  far  background  of  an  image.  Air  temperature  history 
is  required  to  estimate  the  temperature  of  ground  objects.  Ground 
moisture  changes  the  thermal  conductivity  as  well  as  reflectivity 
of  the  earth  surface.  Other  relevant  items  would,  of  course,  be 
included. 
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6.  Background  Data  File 


This  file  only  contains  information  on  the  surface  of  the  DTB . 
It  specifies  the  elevation  of  each  point,  the  surface  properties  of 
the  point,  and  indicates  what  objects,  if  any,  reside  on  that 
point . 

The  surface  properties  of  each  point  will  be  specified  in  the 
file.  These  would  include  surface  texture,  consistency,  and  other 
properties.  Consistency  would  specify  the  surface  type,  i.e., 
grass,  sand,  rock.  Heat  conductance  and  spectral  properties,  such 
as  reflectivity  and  emissivity,  also  would  be  included. 

The  object  indicator  would  be  a  pointer  to  a  specific  station¬ 
ary  object  or  piece  of  stationary  object,  or  vegetation  that 
resides  on  that  point.  Examples  are  given  in  the  object  and  vege¬ 
tation  files. 

7 .  Object  Data  Files 

This  will  be  a  set  of  files  that  contain  information  on  a 
specific  object.  The  location  of  any  objects  would  be  specified  in 
the  background  data  file.  Objects,  as  defined  in  this  paper, 
include  any  man  made  structures  that  might  affect  the  output  of  a 
sensor.  Both  geometric  and  spectral  characteristics  will  be 
included  in  this  file. 

It  is  envisioned  that  object  files  would  be  generated  as 
needed  and  thus  the  collection  would  grow  slowly.  Object  files 
initially  would  take  a  lower  priority  than  vehicle  files.  Some  of 
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the  more  necessary  files  would  describe  buildings,  fences,  rail 
road  tracks,  or  power  lines.  Note  that  there  is  a  need  for  many 
different  building  or  fence  files,  but  these  could  be  generated 
only  when  needed  and  then  saved  for  later  use. 

8.  Vegetation  Data  Files 

The  vegetation  models,  mostly  trees,  probably  will  be  generat¬ 
ed  using  fractal  methods.  Different  tree  types  could  use  different 
fractal  models  or  the  same  models  with  different  input  parameters. 
Due  to  the  computation-intensive  nature  of  fractal  images,  it  is 
hoped  that  a  set  of  trees  of  various  types  could  be  stored  in  a 
data  base  and  recalled  randomly.  This  will  require  a  large  set  of 
trees  and  an  adequate  method  of  storing  the  output.  It  should  be 
noted  that  storage  of  fractal  images  could  be  a  non-trivial 
problem. 

The  vegetation  models  should  include  spectral  and  geometric 
characteristics  in  order  for  the  signature  models  to  estimate  the 
vegetation  observables. 


9.  Vehicle  Structure  Files 

The  vehicle  structure  files  will  contain  all  static  informa¬ 
tion  about  a  given  vehicle.  This  would  include  detailed  vehicle 
geometry,  texture  of  outside  surfaces,  reflectivity  of  outside 
surfaces,  heat  conductance  and  emissivity  of  vehicle  parts,  and 
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miscellaneous  information  such  as  internal  heat  sources  or  fluid 
circulation  which  effects  vehicle  observables.  More  information 
can  be  added  as  its  relevance  becomes  apparent. 

It  is  anticipated  that  the  US  Army  Ballistic  Research  Labora¬ 
tory  (BRL)  target  models,  which  contain  detailed  external  and 
internal  geometric  information,  may  be  used  as  the  starting  point 
for  preparing  the  vehicle  structure  files.  If  needed  information 
were  added  to  these  BRL  models,  they  could  serve  as  vehicle  struc¬ 
ture  files  for  the  DTB . 


10 .  Synthetic  Scene  Generators 

The  designers  of  a  synthetic  scene  generator  for  a  given 
sensor  would  use  the  above  files  to  create  the  sensor  output.  The 
approach  taken  would  be  determined  by  the  designer,  but  there  are 
certain  characteristics  which  would  be  encouraged  for  inclusion  in 
the  DTB. 

A  modular  approach  should  be  used  whenever  possible,  which 
would  facilitate  making  changes  in  the  system,  as  well  as  allow 
mixing  and  matching  of  parts.  For  example,  it  might  be  desirable 
to  use  one  developer's  atmospheric  model  while  using  another  sensor 
model.  In  order  to  make  integration  of  any  component  easier,  it  is 
desirable  that  ongoing  discussion  take  place  between  a  designer  and 
those  integrating  their  components  into  the  DTB. 
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It  is  also  desirable  to  make  each  component  as  flexible  as 
possible.  For  example,  a  module  that  can  handle  both  common  module 
and  focal  plane  array  FLIRs  would  be  preferable  to  a  more  limited 
system.  This  flexibility  would  make  comparison  of  the  two  systems 
easier . 


11.  Imago  Validation  Isauoa 

A  key  factor  in  determining  the  usefulness  of  the  DTB  will  be 
the  realism  of  the  synthetic  sensor  outputs.  Synthetic  images 
should  not  be  used  to  develop  or  test  ATRs  or  to  train  humans  if 
they  are  not  realistic.  The  authors  believe  that  methods  for  evalu¬ 
ation  and  validation  of  synthetic  images  should  be  prepared  prior 
to  testing  or  training  in  order  to  facilitate  attaining  a  consensus 
on  the  validity  of  specific  synthetic  images.  The  issue  of  image 
validation  is  controversial,  therefore,  not  all  the  methods  sug¬ 
gested  here  necessarily  will  be  used. 

The  first  method  of  examination  would  be  a  review  of  the 
sensor  output  by  human  experts.  The  experts  would  look  at  the 
signatures  of  man-made  and  natural  objects  in  the  scene  and  make  a 
subjective  evaluation  of  realism. 

For  synthetic  FLIR  images  only,  a  large  set  of  images  could  be 
generated  representing  several  backgrounds  with  targets  at  a  number 
of  different  ranges  and  temperature  levels.  A  similar  set  of 
images  could  be  generated  on  the  physical  terrain  board,  and  both 
sets  would  be  used  as  input  to  a  set  of  ATRs.  The  scored  output  of 
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the  ATRs  should  be  statistically  equivalent  for  physical  terrain 
board  and  synthetic  images.  The  same  method  could  be  used  to  com¬ 
pare  ATR  performance  on  field  and  synthetic  data,  using  FLIR  input, 
TV,  Ladar,  millimeter  wave  radar,  or  other  sensors,  in  single  or 
multi-sensor  mode. 

Other  approaches  have  been  suggested.  The  previous  method 
could  be  applied  to  human  observers,  instead  of  ATRs,  for  low 
contrast  targets.  For  non-imaging  systems,  such  as  millimeter  wave 
radar,  a  target  signature  correlation  comparison  probably  would  be 
useful.  For  very  high  quality  synthetic  images,  human  observers 
could  be  asked  to  determine  which  images,  from  a  set,  are  synthetic 
and  which  are  real.  Image  metrics  could  be  computed  on  synthetic 
and  real  images,  and  a  comparison  made.  The  issue  of  image  valida¬ 
tion  will  continue  to  be  controversial,  but  it  demands  greater 
attention  if  a  useful  DTB  is  to  be  realized. 
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ABSTRACT 


Synthetic  image  generation  is  essential  for  the  development  of  robust  Automatic  Target 
Recognition  (ATR)  systems.  Although  it  does  not  eliminate  the  need  for  real  data  collection, 
it  offers  flexibility  and  supports  traceability,  parametric  analysis,  and  ATR  performance 
optimization.  This  paper  describes  the  synthetic  image  generation  tools  that  we  have  built 
and  implemented  to  support  the  development  of  ATR’s  and  Multisensor  Fusion  (MSF) 
algorithms.  The  suite  of  simulated  sensors  includes  laser  radar,  millimeter  wave  radar, 
infrared,  and  synthetic  aperture  radar.  These  tools  were  designed  to  simulate  both  target  and 
background,  and  are  based  on  geometric  facet  models.  With  the  facet  model,  objects  are 
assumed  to  be  solid  an'1  a  a  represented  with  triangular  panels  defining  only  tne  outer 
surface.  Such  a  modeling  approach  conveniently  supports  the  requirement  for  concurrent 
and  colocated  imago  generation  of  several  sensors. 

To  simulate  laser  radar  imagery,  target  and  background  facets  are  binned  into  the 
appropriate  angle-angle  and  range  bins.  Speckle,  noise,  and  sensor  response  functions  are 
modeled  in  the  simulator.  Millimeter  wave  turntable  image  simulation  is  similar  to  laser 
radar  simulation  with  the  exception  that  two-dimensional  binning  is  done  in  azimuth  and 
range. 

Infrared  images  are  generated  using  the  same  target  geometric  model  and  assigning 
thermal  grey-scale  values  to  the  facets.  Thermal  assignments  are  based  on  empirical  data 
derived  from  real  imagery  and  can  be  made  by  both  manual  and  automatic  techniques. 
Atmospheric,  battlefield,  and  sensor  effects  are  added  using  the  Thcom  Thermal  Image 
Model. 

These  synthetic  image  tools  have  been  used  to  support  the  development  of  multisensor 
fusion  for  automatic  target  recognition  and  search  algorithms.  This  capability  provides  a 
cost-effective  means  of  generating  a  sufficient  number  of  samples  for  statistics-based 
algorithms,  and  the  synthetic  targets  inherently  lend  themselves  to  the  development  of 
model-based  algorithms  as  well. 


*  The  work  reported  in  this  paper  was  performed  under  Northrop  Corporation  Independent  Research  and 
Development  (IR&D)  funding. 


1.0  Introduction 


The  collection  of  real  data  is  very  expensive  and  time  consuming.  Although  the  use  of 
synthetic  data  does  not  eliminate  the  need  for  real  data,  sensor  and  target  signature  models 
have  a  niche  in  the  development  of  statistics-based,  model-based,  and  Multisensor  Fusion 
(MSF)  Automatic  Ihrget  Recognizer  (ATR)  algorithms.  In  the  following  paragraphs  we 
establish  the  needs  for  multisensor  modeling  for  ATR  algorithm  development  and 
performance  evaluation. 


1.1  Statistical  Pattern  Recognition 

Many  existing  ATR’s  use  statistics-based  algorithms  to  detect  and  classify  targets 
found  in  Forward  Looking  Infrared  (FUR)  imagery.  Feature  vectors,  assigned  to  potential 
targets,  form  a  feature  vector  space  that  is  separated  into  regions,  and  each  region  is  assigned 
a  probability  based  on  whether  or  not  a  feature  vector  (object)  within  that  region  is  a  target. 
Since  measured  features  are  susceptible  to  noise,  statistical  trends  over  a  large  number  of 
images  are  more  meaningful  than  a  few  measurements  made  from  a  limited  number  of 
images.  The  large  data  base  requirements  of  this  approach  can  be  fulfilled  by  the 
augmentation  of  real  imagery  with  synthetic  imagery. 


1.2  Model-Based  Target  Recognition 

The  current  trend  in  ATR  development  is  toward  the  use  of  model-based  algorithms. 
The  inherent  nature  of  synthetic  targets  makes  them  the  comers  tone  of  a  model-based 
technique.  At  the  very  least,  “ideal”  (uncorrupted)  synthetic  target  signatures  act  as  the 
reference  model  to  which  sensed  target  signatures  are  aligned  and  compared.  Additional 
synthetic  target  signatures  can  be  corrupted  and  convolved  with  point  spread  functions  and 
used  as  the  input  or  sensed  image  to  the  model-based  algorithm. 
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13  Development  of  Multiple  Sensor  Algorithms 

The  limited  availability  of  multisensor  data  can  hinder  the  development  of  algorithms 
for  sensor  suites.  Without  a  sufficient  data  base,  algorithms  are  sometimes  limited  in 
performance  by  the  inability  to  test  and  refine  development.  Data  generated  from  different 
sensor  models  provide  registered  imagery  on  which  even  pixel-level  fusion  can  be 
investigated.  Additionally,  with  the  use  of  a  multisensor  data  base,  studies  can  be  conducted 
to  determine  the  best  combinations  of  sensor  types  for  directed  mission  and  scenario 
requirements. 

1.4  Performance  Evaluation 

To  adequately  assess  the  performance  of  a  single  sensor  or  multisensor  ATR, 
exhaustive  parametric  studies  are  needed.  To  this  end,  some  parameters  within  the  images 
are  fixed  while  performance  is  measured  under  various  conditions  of  other  parameters.  For 
instance,  performance  of  an  algorithm  can  be  measured  with  respect  to  changes  in  target 
orientation  or  inaccuracies  in  range  estimations.  It  is  important  to  be  able  to  fix  specific 
parameters  within  a  data  base  while  testing  algorithms’  robustness  with  respect  to  other 
parameters.  Images  generated  from  a  synthetic  sensor  modeling  system  are  easy  to  “ground 
truth”  and  provide  a  large  number  of  target  signatures  at  any  specified  orientation  and 
thermal  state. 

2.0  Techniques  for  Multisensor  Modeling  Sensor  Modeling 

Using  a  triangular  facet  model  as  a  basis,  we  have  developed  and  integrated  a  system 
that  models  CO2  laser  radar,  Millimeter  Wave  (MMW)  Radar,  Synthetic  Aperture  Radar 
(SAR),  and  Infrared  target  signatures. 

2.1  CO2  Laser  Radar  Model 

The  imaging  model  used  is  the  point  projection  model  [1]  that  approximates  the 
transformation  performed  by  our  eye  and  many  imaging  devices  on  a  visual  scene.  The 
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imaging  process  consists  of  passing  the  scene  through  a  pinhole  (focus)  and  projecting  the 
image  onto  an  image  plane.  This  is  depicted  for  the  triangular  panel  in  Figure  1.  The  origin 
“o”  is  the  location  of  the  focus. 


The  object  represented  is  assumed  to  be  solid  with  the  triangular  panels  defining  only 
the  outer  surface.  These  triangular  panels  therefore  have  intersections  only  at  their  edges. 
We  have  adopted  a  convention  for  facet  representation  whereby  the  information  stored  for 
each  triangular  facet  consists  of  the  three-dimensional  coordinates  of  the  three  comer  points. 
These  points  are  ordered  in  a  counterclockwise  direction  corresponding  to  a  condition  where 
the  “outer”  surface  normal  of  the  facet  has  a  positive  component  in  the  direction  of  the  viewer 
as  shown  in  Figure  2  (the  normal  will  be  pointed  directly  at  the  viewer  if  such  component  of 
the  unit  normal  is  1).  The  “outer”  unit  normal  vector  is  then  determined  by: 
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(1) 


N  -  { [  (y2  -  yi)  (23  -  z2>  -  (z2  -  zi)  to  -  y2)]  ' 

+  [  (z2  -  zi  (x3  x  2)  -  (X2  -  xi)  (z3  -  z2>]  I  ^ 

+  [  (X2  -  xi)  (y3  -  y2)  -  (y2  ~  Yi)  (x3  -  x2)l  * 

n  =N/|iN||  (2) 

n  «nx  1  +ny;  +  nz  £  (3) 

where  (x,,  yj,  Zj)  are  the  coordinates  of  the  triangle’s  comers.  N  is  the  cross  product  of 
sides  a  and  b  as  shown  in  Figure  2. 


Figure  2:  Panel  Normal 


Shadowing  of  the  facets  is  determined  analytically  [2].  Simulation  of  a  laser  radar 
image  then  proceeds  by  binning  facets  or  sections  of  facets  visible  to  the  sensor  in  the 
appropriate  angle-angle  and  range  bins  [3],  Speckle,  noise,  and  the  sensor  response  function 
are  modeled  by  the  simulator  (Figure  3). 


A  simulated  2-D  angle-angle  (no-range)  laser  radar  image  of  a  T-62  tank  with 
resolution  of  6  inches  at  viewing  angles  of  30  degrees  in  elevation  and  120  degrees  in  azimuth 
is  shown  in  Figure  4.  The  facet  data,  acquired  from  the  U.  S.  Air  Force  Armament  Technology 
Laboratory  at  Eglin  Air  Force  Base,  consists  of  about  1,100  facets  defining  the  outer  surface 


27 


Figure  3.  Laser  Radar  Target  Signature  Generation  Procedure 


of  the  T-62  tank.  A  color-coded  simulated  range-angle  image  at  the  same  viewing  angles  is 
shown  in  Figure  5. 

The  3-D  information  (angle-angle-range)  can  be  rotated  to  provide  a  planar  view  of 
the  target  as  shown  in  Figure  6.  The  shadowing  caused  by  the  turret  at  this  aspect  angle  is 
clearly  visible. 


Figure  4.  Angle-Angle  Image,  T-62,  Soviet  Tank 
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Figure  5.  Range-Angle  Image,  T-62  Soviet  Tank 


Figure  6.  Rotated  Image  T-62  Soviet  Tank 


2.2  MMW  Sensor  Model 

The  modeling  of  MMW  sensor  imagery  consists  of  binning  facets  or  parts  of  facets  into 
the  appropriate  angle-range  bins  (Figure  7).  The  cross  section  is  then  calculated  for  each  bin. 

Physical  optics  and  edge  diffraction  are  currently  modeled  in  the  algorithm  for  calculating 
cross  section.  Polarization  signatures  can  be  modeled  for  each  bin,  given  appropriate 
modeling  tools  for  calculating  the  polarization  response  of  a  collection  of  facets  in  a  bin.  The 
facet  representation  and  shadowing  techniques  are  described  in  the  laser  radar  model 
section.  A  comparison  of  MMW  Imaging  SAR  (ISAR)  simulations  with  MMW  turntable 
data  on  a  T-62  tank  is  shown  in  Figure  8. 
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Figure  7.  MMW  Target  Gene  ration  Procedure 
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Figure  8.  MMW  llirntable  Data  vs.  Simulation,  T-62,  Soviet  Tank 

23  SAR  Model 

We  have  developed  a  combined  target  and  background  signature  simulator  to  support 
development  of  SAR  ATR  algorithms  [3],  The  calculation  of  the  SAR  image  of  a  target, 
depicted  in  Figure  9  proceeds  as  follows:  (1)  The  three-dimensional  surface  boundary  of  a 
target  is  represented  by  facets  as  described  in  paragraph  2.1;  (2)  the  facets  or  portions  of 
facets  are  binned  into  their  appropriate  SAR  image  bins;  (3)  shadowed  facets  or  parts  of  facets 
are  determined  and  eliminated;  and  (4)  the  electromagnetic  field  scattered  by  each  facet  in  an 
image  is  calculated  using  the  formalism  of  physical  optics  and  diffraction  as  documented  in 
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[4].  Shown  in  Figure  10  is  a  comparison  of  measured  SAR  data  of  a  B52  airplane 
simulated  imagery. 


Figure  9.  SAR  Image  Generation  Procedure 


SIMULATED  VISIBLE  IMAGERY 


SAR IMAGE 


SIMULATED  SAR  IMAGE 


Figure  10.  Measured  and  Simulated  B52  SAR  Images 


The  background  signature  simulation  used  in  our  simulator  is  based  on  the  Radar 
Image  Simulation  (RIS)  software  developed  by  the  Remote  Sensing  Laboratory  at  the 
University  of  Kansas,  Lawrence.  Input  to  the  simulator  consists  of  terrain  elevation  maps, 
lands  use  maps,  and  texture  maps.  A  final  SAR  target  and  background  image  is  generated  by 
embedding  the  target  into  the  background  image  and  applying  system  noise.  Shown  in  Figure 
1 1  is  a  simulated  image  showing  military  vehicles  positioned  in  a  clearing. 


2.4  IR  Model 


The  current  implementation  of  the  synthetic  infrared  model  places  an  infrared  target 
in  either  a  real  or  synthetic  background.  Grey-level  values  are  assigned  to  the  projections  of 
the  three-dimensional  target  facets  onto  the  two-dimensional  image  plane.  Various 
approaches  can  be  employed  to  assign  thermal  states  to  the  target  facets.  The  current 
implementation  (shown  in  Figure  12)  assigns  pixel  values  by  examining  pixel  values  in  real 
imagery  and  assigning  those  values  to  the  projected  facets  of  a  model  target  of  similar 
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Figure  12.  Thermal  Image  Generation  Procedure 

orientation.  In  algorithm  development,  the  contrast  between  a  target  and  background  is 
often  the  most  important  feature  of  the  infrared  target  signature.  With  this  in  mind,  pixel 
values  are  also  assigned  to  the  target  model  such  that  the  relative  grey  values  of  the  synthetic 
image  closely  match  those  of  a  real  image  under  the  same  set  of  conditions. 

The  modeled  thermal  signature  of  the  target  is  placed  in  a  real  or  synthetic 
background  and  a  point  spread  function  is  convolved  over  the  final  image  to  mimic  sensor 
effects.  Also,  the  final  image  can  be  used  by  the  TACOM  Thermal  Imaging  Model 
(TTIM — OptiMetrics,  Inc.)  to  model  atmospheric  and  battlefield  effects. 

3.0  Conclusions  and  Recommendations 

Synthetic  image  generation  is  essential  for  the  development  of  statistics-based  and 
model-based  ATR  algorithms.  The  target  descriptions  used  by  synthetic  image  systems 
should  be  designed  to  support  target  signature  modeling  of  multiple  sensors.  This  provides  a 
basis  for  the  generation  of  registered  synthetic  imagery  from  multiple  sensors  and  allows  an 
algorithm  developer  to  perform  a  well-defined  parametric  analysis  of  ATR  performance. 
The  effects  of  image  and  sensor  degradation  as  well  as  deviations  from  predicted  mission 
scenarios  can  be  studied  and  quantified  using  synthetic  imagery. 
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There  are  several  different  modules  for  creating  synthetic  image  data,  but  an  overall, 
user-friendly  system  or  environment  needs  to  be  developed  to  fully  utilize  the  individual 
packages.  Coordination  between  government  and  industry  would  help  accomplish  this  and 
would  reduce  duplication  of  effort.  Government-defined  mission  and  validation 
requirements  will  allow  industry  to  develop  synthetic  sensor  model  and  terrain  data  that  will 
boost  ATR  technology  and  benefit  the  entire  community. 
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ABSTRACT 

The  advantages  of  using  a  synthetic  database  for  the  development  testing,  and  evaluation  of  Automatic  Target 
Recognition  algorithms  are  well  recognized.  Real  multi -sensor  scene  combinations  reflecting  operational  requirements, 
missions,  and  targets  are  unavailable  and  economically  difficult  if  not  impossible  to  obtain.  This  paper  describes  Texas 
Instruments  (TI’s)  Synthetic  Multi-Sensor  (IR,  TV,  Laser  Radar)  Image  Generation  System  which  has  been  developed  to 
address  this  database  problem.  The  system  described  has  the  capability  of  generating  single  image  frames  or  image 
sequences  based  upon  scenarios  derived  directly  from  DMA  digital  map  data  The  major  components  of  the  system 
include  scene  rendering,  sensor,  atmospheric  environment  and  user  interface  modules.  The  generation  of  synthetic 
backgrounds  (trees)  utilizing  fractal  techniques  is  also  considered. 

1.  INTRODUCTION 

The  development  of  robust  automatic  target  recognition  (ATR)  algorithms  depends  critically  on  an  adequate  database 
for  testing  them.  At  the  present  time  there  is  a  significant  lack  of  real,  boresighted,  multi-sensor  imagery  available  for 
this  purpose.  Furthermore,  the  acquisition  of  such  data  is  difficult  and  expensive  in  the  variety  of  environmental  and  target 
conditions  that  are  needed.  In  1988  Texas  Instruments  completed  the  development  and  validation  of  a  multi-sensor  hybrid 
image  generation  system  to  extend  existing  real  databases  by  inserting  synthetic  targets  in  real  image  backgrounds.  It  was 
an  extension  of  the  Defense  Advanced  Projects  Agency  (DARPA)  Characterization  of  Algorithms  for  Passive  Infrared 
(CAPIR)  program'.  This  system  has  the  capability  of  modeling  various  sensors  and  controlling  image  parameters  of 
importance  to  ATR  algorithms  such  as  signal-to- noise  ratio,  target-to- background  contrast  ratio,  target  detail  (variance  of 
target  intensities),  and  clutter  content  (IR  and  TV)  in  image  backgrounds.  However,  this  hybrid  system  is  not  adequate 
to  support  autonomous,  ATR  system  smart  search  and  end-to-end,  mission  simulations.  Hence,  a  synthetic,  muiti-sensor(IR, 


Cockcroft,  Alan,  Krebaum,  Ken,  Thomson,  Chris  J.,  "Final  Report,  Characterization  of  Algorithms  for 
Pteive  Infrared  (CAPIR),"  Report  No.  U2-85301-F.  Center  for  Night  Vision  &  Electro-Optics,  Advanced 
Concepts  Division,  Fort  Belvoir,  VA  22060;  31  January  1988. 
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TV,  Laser  Radar)  image  generation  system  utilising  Defense  Mapping  Agency  data  with  modeling  of  scene  backgrounds, 
sensors,  and  environmental  effects  was  created.  Particular  emphasis  has  been  placed  on  those  features  necessary  to  support 
ATR  algorithm  development 

2.  SYNTHETIC  SCENE  RENDERING 

The  synthetic  image  generation  system  has  been  developed  to  produce  single  image  frames  or  image  sequences  of 
IR,  TV,  and/or  laser  radar  images  as  shown  in  Figure  1.  The  terrain  elevation  and  background  cultural  data  are  derived 
from  Defense  Mapping  Agency  (DMA)  Digital  Terrain  Elevation  Data  (DTED)  and  Digital  Feature  Analysis  Data  (DFAD). 


IR  TV  Laser  Radar 

Figure  1.  Synthetic  Imagery 

The  DMA  elevation  data  are  interpolated  using  a  fractal  interpolation  technique  to  achieve  a  realistic  appearing  terrain  at 
the  resolution  of  the  sensors  being  simulated.  Fractal  techniques  have  also  been  used  to  create  individual  background  trees 
of  various  types  and  forested  regions.  Figure  2  presents  a  block  diagram  of  the  synthetic  image  generation  system 
currently  in  operation. 

Three  types  of  data  are  utilized:  terrain  dam  at  a  base  resolution  determined  by  the  DMA  elevation  data  used;  class 
map  data  containing  material  type  data  derived  from  DMA  feature  analysis  data,  registered  with  the  terrain  data,  and  stored 
in  quad-tree  form;  and  object/target  data  stored  as  facet  and  point  lists.  In  general,  a  large  portion  of  the  scenario  is  not 
within  a  sensor’s  field-of-view  (FOV)  during  a  single  snapshot.  Hence,  terrain  data  are  initially  clipped  to  a  subset  of 
the  overall  terrain  area  to  reduce  the  amount  of  data  to  be  processed.  Exact  terrain  points  within  the  projected  sensor 
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footprint  on  the  ground  are  determined  and  are  subsequently  used  in  further  processing.  In  addition,  objects  located  within 
these  active  FOV  terrain  areas  are  determined  and  tagged  for  processing.  The  computational  time  required  for  generating 
an  image  is  further  reduced  by  utilizing  user  specified  near  and  far  clipping  planes  outside  of  which  objects  and  terrain 
are  not  processed. 

All  objects  specified  within  a  senses'’ s  field-of-view  are  translated,  rotated,  and  scaled  to  the  desired  location  and 
orientation  in  world  coordinates.  A  transformation  from  the  world  coordinate  system  to  the  sensor  viewpoint  is  performed 
and  a  perspective  transformation  maps  the  object  and  terrain  data  into  a  two-dimensional  sensor  image  plane  view.  A 
Z- buffer  hidden  surface  algorithm  is  then  used  to  calculate  which  surfaces  are  blocked  from  view  by  other  surfaces.  The 
output  contains  the  visible  surfaces  as  they  would  appear  to  the  sensor  with  a  geometric  sampling  resolution  sufficient  for 
the  application  of  subsequent  antialiasing  processing. 
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Figure  2.  Synthetic  Scene  Generation  System  Block  Diagram 
Greyshades  are  now  applied  to  the  two-dimensional  sensor  image.  The  shading  of  the  IR  sensor  imagery  currently 
depends  upon  user  determined  shades  with  solar  effects  not  implemented  (future  work  will  incorporate  thermal  modeling 
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from  first  principles).  Attached  to  each  object  or  terrain  model  point  is  a  property  which  can  be  used  as  a  greyshade  or 
related  to  a  greyshade  (i.e,  material  type  from  the  class  map  data).  The  area  between  these  points  is  shaded  using 
Gouraud  (intensity  interpolated)  shading.  Greyshading  for  TV  sensor  simulation  consists  of  ambient  and  diffuse 
components  (specular  shading  is  not  currently  implemented)  with  coefficient  lookup  tables  used  for  different  material  types. 

Laser  radar  sensor  simulation  is  obtained  by  generating  a  range  image  at  the  required  sensor  pixel  resolution  and 
applying  the  necessary  sensor  effects  to  simulate  a  particular  laser  radar  sensor. 

Subpixel  areas  are  finally  averaged  to  reduce  antialiasing  artifacts  and  to  obtain  the  final  sensor  imagery. 

The  scene  rendering  system  described  has  a  wide  variety  of  configurations  available  to  the  user  for  multi-sensor  image 
generation.  The  sensor  type  (TV,  IR,  and  Laser  Radar),  sensor  parameters  (FOV  and  resolution),  and  location/orientation 
(latitude,  longitude,  altitude,  beading,  depression  angle,  and  roll)  are  all  user  selectable.  Also,  multiple  sensors  (sensor 
types,  wide/narrow  FOV’s,  etc)  may  be  included  in  a  sequence  with  any  sensor  capable  of  being  selected  for  a  particular 
frame.  Objects  may  be  shaded  with  unique  grey  shades  in  a  sensor’s  FOV  (e.g.,  an  M-60  can  be  shaded  with  a  single 
greyshade,  each  type  of  tree  with  its  own  different  greyshade,  and  various  material  types  of  the  terrain  with  still  different 
grey  shades).  This  can  be  useful  for  determining  ground  truth  for  the  testing  of  ATR  segmentation  algorithms. 

3.  SYNTHETIC  BACKGROUND  GENERATION 

Synthetic  background  generation  methods  utilizing  fractal  techniques  have  been  developed  to  increase  scene  detail  with 
emphasis  on  the  generation  of  individual  tree  types,  tree  lines,  and  forests.  These  methods  have  been  severely  constrained 
because  of  the  requirement  to  generate  synthetic  scenes  as  fast  as  is  practically  possible. 

Two  types  of  individual  trees  (deciduous  and  coniferous)  have  been  modeled  as  well  as  individual  trees  without  leaves. 
For  trees  without  leaves  a  recursive  procedure  is  used  with  a  tree  node  defined  as  a  branch  (stem)  with  one  or  mote  tree 
nodes  attached  and  with  each  tree  node  generated  by  means  of  a  3x3  linear  transformation.  The  generation  process  is 
terminated  once  the  branches  (stems)  are  small  enough.  The  variable  parameters  of  the  algorithm  are:  stem  diameter  at 
base  of  the  tree,  initial  stem  length,  minimum  stem  length,  and  number  of  branches  per  branching  point  The  branches 
and  stems  are  modeled  as  elongated  three-sided  pyramids.  Additional  details  can  be  found  in  the  paper2  referenced 
below. 

Trees  with  leaves  are  generated  using  a  fractal  initiator/generator  technique.  This  is  accomplished  in  three  steps.  The 
first  step  is  to  use  fractal  techniques  to  generate  the  outline  of  the  tree  object  as  viewed  from  uniformly  spaced  directions 

2  Oppenheimer,  Peter  E.,  "Real  Time  Design  and  Animation  of  Fractal  Plants  and  Trees,”  SIGGRAPH  'j$6 

Conference  Proceedings.  Vol.  20,  No.  4,  August  1986;  pg.  55-64. 


38 


around  it  The  number  of  such  viewing  directions  is  an  algorithm  input.  This  set  of  two-dimensional  half-planes 
essentially  consists  of  tree  side  views  as  seen  from  appropriate  directions.  The  second  step  is  to  orient  each  half-plane 
in  three-dimensional  object  space  at  the  appropriate  rotation  angle  about  the  tree  axis.  Finally  the  vertices  of  the  straight 
lines  making  up  each  half-plane  tree  outline  are  connected  together  to  form  point  and  facet  lists  defining  the  tree  model 
which  can  then  be  rendered  by  the  scene  rendering  software  after  having  been  transformed  from  object  to  world  coordinate 
space.  Examples  of  the  results  obtained  are  shown  in  Figure  3. 

Since  tree  lines  define  the  edges  of  forests  where  partially  obscured  or  totally  obscured  target  objects  may  be  located, 
scene  realism  is  an  important  issue  for  the  development  and  testing  of  advanced  ATR  algorithms  which  are  being  designed 
to  locate  targets  in  these  regions.  The  fractal  technique  used  to  model  individual  trees  with  leaves  as  described  above 
has  been  applied  to  large  groups  of  trees.  Tree  lines  have  been  simulated  by  randomly  placing  individual  trees  along  the 
defined  tree  lines.  Forest  regions  have  been  modeled  by  randomly  placing  high-detailed  trees  along  its  edge  (tree  lines) 
and  filling  these  regions  with  low-detailed  trees  which  only  have  their  top  halves  modeled.  This  is  possible  since  only 
those  approximating  facets  within  a  line  of  sight  are  visible. 


Figure  3.  Synthetic  Tree  Imagery 

4.  SENSOR  AND  ATMOSPHERIC  ENVIRONMENT  MODELING 
Sensor  models  for  IR,  TV,  and  laser  radar  sensors  were  developed  for  use  with  the  Synthetic  Multi-Sensor  Image 
Generation  System.  They  have  been  specially  designed  for  speed  of  execution  and  hence  have  been  simplified  from  those 
that  were  developed  for  the  hybrid  image  generation  system  mentioned  in  the  introductory  section  of  this  paper.  A  two- 
dimensional  Gaussian  point-spread  function  is  used  to  approximate  the  effects  of  optical  diffraction  and  blur,  detector  and 
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LED  spatial  response,  and  stabilization  jitter  for  both  the  IR  and  TV  sensors.  The  use  of  a  Gaussian  point-spread  function 
greatly  simplifies  both  the  computational  process  as  well  as  the  modeling  process,  since  the  effort  in  determining  all  the 
possible  distortions  and  their  parameters  is  replaced  by  the  simple  determination  of  tin  variance  of  the  Gaussian  estimate 
of  the  system  response.  The  final  output  images  of  the  IR  and  TV  sensors  are  obtained  by  multiplying  the  spatial  MTF 
of  each  sensor  type  (IR  and  TV)  by  the  FFT  of  the  pre-MTF  IR  or  TV  image,  adding  colored  noise  appropriately  matched 
to  the  sensor’s  noise  characteristics,  and  calculating  the  inverse  FFT  for  the  IR  and  TV  images.  The  range  image  which 
is  also  obtained  as  an  output  from  the  scene  generation  system  is  currently  processed  to  simulate  signal  dropouts  and  then 
divided  into  range  bands  or  used  as  an  absolute  range  image  depending  upon  the  laser  radar  sensor  being  simulated. 

In  general,  meteorological  conditions  affect  the  atmospheric  transmission  of  infrared  energy  from  background,  target, 
and  object  scene  elements  to  a  given  sensor.  Knowing  relevant  weather  information,  one  can  predict  the  atmospheric 
transmission  using  the  LOWTRAN-7  model.  This  atmospheric  transmission  is  specified  in  units  of  transmission  per 
kilometer.  This  factor  in  conjunction  with  die  range  value  associated  with  each  image  pixel  is  used  to  take  into  account 
atmospheric  effects. 

5.  USER  INTERFACE 

The  development  of  an  operatorAiser  interface  which  allows  an  operator  or  user  to  control  easily  the  operation  of  the 
synthetic  image  generation  system  and  to  create,  modify  and  control  an  associated  synthetic  scene  scenario  is  of  utmost 
importance  in  achieving  a  successful  capability  to  generate  synthetic  imagery.  The  creation  of  scene  scenarios  is  extremely 
labor  intensive  unless  adequate  scenario  creation  and  editing  facilities  are  available.  Figure  4  shows  the  VAX  computer 
interface  menu  that  has  been  developed.  The  right-hand  portion  of  the  menu  corresponds  to  the  VT-100  keypad  with  the 
individual  keypad  display  areas  highlighted  when  the  associated  key  has  been  activated.  Appropriate  responses  are 
requested  in  the  Information  Area  on  the  left-hand  area  of  the  display.  An  error  message  line  is  utilized  on  the  bottom 
of  the  display  when  errors  occur.  The  following  operator  functions  have  been  implemented:  the  specification  of  scenario 
flight  paths  and  sensor  parameters;  the  addition,  deletion,  or  replacement  of  targets  and  objects;  and  the  graphical  display 
of  the  scenario  map  at  various  scales  with  an  indication  of  the  location  and  type  of  cultural  features  and  objects  in  the 
scenario.  The  interface  also  allows  the  specification  of  the  simulated  rate  of  image  frame  generation  for  each  sensor 
and  displays  an  indication  of  a  sensor’s  fieid-of-view  along  the  specified  flight  path.  In  addition,  an  AutoCad  10,  PC- 
based  software  system  is  used  to  develop  target  and  object  models  for  the  synthetic  image  generation  system.  This  system 
also  allows  target  and  object  models  to  be  imported  through  the  use  of  industry  standard  graphics  interchange  formats. 
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In  this  manner  the  large  number  of  various  synthetic  targets  and  objects  needed  to  support  the  development  of  advanced 
ATR  algorithms  is  cost  effectively  provided. 


Synthetic  Image  Generation 
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Figure  4.  User  Interface  Menu 


6.  CONCLUSIONS  AND  COMMENTS 

The  Synthetic,  Multi-Sensor  Image  Generation  System  developed  by  Texas  Instruments  for  the  development,  testing, 
and  evaluation  of  ATR  algorithms  has  been  described.  It  is  envisioned  that  this  system  will  be  most  useful  for  supporting 
the  smart  search  and  end-to-end,  mission  simulations  of  autonomous,  advanced  ATR  algorithms.  At  this  point  in  time, 
it  is  clear  that  synthetically  generated  multi-sensor  imagery  will  not  replace  real  imagery,  but  it  is  believed  that  use  of 
the  system  described  in  this  paper  will  provide  valuable  insights  not  otherwise  obtainable  during  the  advanced  ATR 
algorithm  development  process. 
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ABSTRACT 


The  complexity  of  real-world  scenes  makes  faithful  digital  simulation  computa¬ 
tionally  expensive  unless  efficient  modeling  and  image  generation  techniques  axe 
applied.  The  large  volumes  of  data  required  to  represent  terrain  and  natural  sur¬ 
face  features  make  storage,  access,  and  processing  costly.  Simplifying  the  data  to 
alleviate  computation  or  merely  overlaying  an  image  onto  terrain  data  will  not 
provide  the  3-D  clutter  and  obscuration  necessary  to  test  target  recognition  fully. 

Grumman  Data  Systems  has  developed  efficient  digital  technology  to  simulate 
complex  and  realistic  3-D  scenes  into  which  specific  targets  can  be  inserted.  The 
background  scenes,  including  terrain,  trees  and  clouds,  can  be  generated  entirely 
from  mathematical  functions  or  can  include  real  terrain  and  feature  data 
enhanced  by  such  functions.  Terrain  is  modeled  by  polygons  fitted  to  a  mesh  of 
elevation  values.  Trees  are  modeled  by  textured  quadric  surfaces  positioned  on 
the  terrain  procedurally.  A  dynamic  cloud  model  simulates  cumulus  cloud 
development  as  well  as  smoke  screens  by  manipulating  ellipsoids  and  controlling 
textural  translucence.  Convincing  tree  and  cloud  shadows  are  simulated 
efficiently,  enhancing  the  realism  of  background  clutter  and  obscuration. 

Levels  of  detail  dependent  on  the  view  point  are  generated  automatically  for  the 
terrain,  trees,  and  clouds.  This  tailors  the  image  content  to  the  viewer’s  percep¬ 
tion,  alleviating  image  computation  by  greatly  reducing  the  amount  of  geometric 
data  required  to  model  distant  scene  features.  In  addition,  detail  on  terrain,  trees, 
and  clouds  is  simulated  by  texture  to  provide  natural  complexity  without  adding 
to  the  geometric  data  base. 


43 


1.  INTRODUCTION 


The  digital/electronic  terrain  board  (DETB)  will  be  a  useful  tool  for  a  variety  of 
applications  which  include  acquiring  or  recognizing  targets  in  cluttered  back¬ 
grounds.  The  control  provided  by  computer  simulation  offers  far  greater  flexibil¬ 
ity  than  physical  model  boards,  allowing  changes  in  targets  and  environments 
with  relative  ease  compared  to  physical  models.  Unfortunately,  the  real  world  is 
extremely  complex  and  includes  a  wide  variety  of  natural  detail  difficult  to  simu¬ 
late  with  traditional  computer  graphics  techniques.  Valid  simulation  of  natural 
environments  must  include  a  representation  of  this  natural  clutter  and  obscura¬ 
tion. 

Computer  graphics  technology  has  made  impressive  advances  toward  realistic 
simulation  of  natural  scenes.  Two  techniques  in  particular  have  been  very  success¬ 
ful.  Fractal  surfaces  [2,5]  allow  representation  of  terrain  with  a  high  level  of 
three-dimensional  detail.  But  the  standard  implementation  of  fractals  entails  the 
creation  of  a  terrain  model  composed  of  hundreds  of  thousands  of  polygons  or 
elevation  points,  necessitating  a  large  geometric  data  base  that  requires  heavy 
computation  loads  for  rendering.  The  second  innovative  technique  is  the  overlay¬ 
ing  of  high  resolution  imagery  on  lower  resolution  terrain  data.  This  technique 
provides  photographic  realism  at  high  altitude  views,  but  fails  to  provide  three- 
dimensional  structure  at  low  altitudes. 

Grumman  Data  Systems  (GDS)  has  taken  a  different  approach  to  the  simulation 
of  natural  scenes  to  provide  realistic  target  backgrounds  economically.  Our 
approach  is  to  minimize  the  geometric  data  base  and  represent  natural  detail  with 
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texturing.  Our  technique  extends  research  initiated  at  the  Grumman  Corporate 
Research  Center  (CRC)[3].  This  initial  work  used  quadric  surfaces  to  model  scene 
features  and  added  detail  by  means  of  a  texturing  function  which  modulated  sur¬ 
face  shading  intensity  and  object  translucence.  At  GDS  we  have  extended  the 
technique  of  functional  texturing  to  a  more  comprehensive  technique  of  functional 
modeling  by  using  the  "texturing"  function  to  define  large  scale  natural  structure. 
This  greatly  facilitates  the  construction  of  arbitrary  natural  scenes  covering  large 
areas.  It  also  provides  a  greater  degree  of  realism  because  of  the  natural  quality 
produced  by  the  functions.  We  have  also  incorporated  the  function  into  a 
dynamic  cloud  model  to  enhance  the  scene  model  with  time-varying  obscuration 
from  atmospheric  clouds  and  smoke  screens. 

A  second  extension  we  have  made  to  the  original  CRC  research  is  the  use  of 
hierarchical  data  structures,  to  reduce  data  volume  by  matching  resolution  to 
view  range. 

2.  A  FUNCTIONAL  MODEL  FOR  NATURAL  SCENES 

Our  functional  scene  model  is  currently  designed  to  provide  three  elements  critical 
for  target  backgrounds: 

1.  Terrain  model 

2.  Clutter 

3.  Obscuration 

The  scene  model  is  generated  using  spectral  functions,  i.e.  functions  defined  by 
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their  frequency  content.  Our  basic  spectral  function  is  composed  of  sums  of 
cosines  with  increasing  frequency  [4].  Our  2-D  function  uses  two  sums  of  cosines. 
The  cosines  of  the  first  sum  are  variables  of  the  south-to-north  ground  coordinate; 
and  the  cosines  of  the  second  sum  are  functions  of  the  west-to-east  ground  coordi¬ 
nate.  Our  3-D  function  includes  a  third  sum  of  cosines  dependent  on  the  vertical 
coordinate. 

By  appropriate  definition  of  the  frequency  content  of  the  functions,  we  are  able  to 
model  large  scale  structure  using  no  more  than  7  cosines  per  sum.  We  can  add 
natural  detail  using  functions  with  the  same  number  of  cosines  but  of  higher  fre¬ 
quency. 

2.1  Terrain 

We  generate  our  terrain  model  by  defining  a  horizontal  grid  with  specified  grid 
spacing.  We  select  parameters  for  our  spectral  function  to  define  the  desired  ter¬ 
rain  characteristics.  A  wide  spectral  bandwidth  will  produce  rugged  terrain,  while 
attenuating  high  frequencies  produces  smooth  terrain.  We  compute  the  function 
at  each  grid  point  and  define  the  elevation  at  that  point  to  be  the  resulting  func¬ 
tion  value.  We  then  triangulate  each  quadrilateral  to  produce  a  geometric  terrain 
data  base  of  triangular  polygons. 

This  low-frequency  geometric  terrain  data  is  enhanced  by  adding  natural  detail 
with  functions  having  higher  frequency  content.  Color  variations  are  added  by 
modulating  the  basic  terrain  color,  and  topographical  detail  is  added  by  modulat¬ 
ing  the  surface  normals  [l].  The  resulting  imagery  is  comparable  to  that  produced 
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by  fractal  surfaces  modeled  with  10  to  100  times  more  polygons.  Because  the  func¬ 
tional  detail  is  applied  in  2-D  image  space,  it  requires  significantly  less  computa¬ 
tion  than  geometric  (polygon)  detail  which  must  be  processed  in  3-D  scene  space. 

2.2  Clutter 

The  most  prevalent  source  of  clutter  in  natural  scenes  is  vegetation,  particularly 
trees.  To  avoid  the  geometric  complexity  normally  associated  with  trees,  we  model 
such  features  with  simple  curved  surfaces,  ellipsoids.  To  facilitate  generation  of 
large  clusters  of  trees,  we  use  our  2-D  function  computed  on  a  fine  grid  on  the 
terrain’s  surface.  Applying  a  threshold  to  the  function  values  isolates  regions 
where  tree  clusters  exist.  At  each  tree  grid  point  within  such  a  region,  we  place  a 
tree  ellipsoid  with  small  random  variations  in  position,  size  and  color.  To  assure 
consistency  in  the  scene  when  viewed  from  different  eye  points,  the  random  varia¬ 
tions  are  keyed  to  the  tree  grid  position. 

We  add  natural  detail  to  each  tree  ellipsoid  by  manipulating  its  surface  shading 
intensity  and  translucence  using  our  3-D  spectral  function  with  high  frequencies. 
A  natural  foliage  look  is  achieved  by  increasing  the  translucence  at  the  ellipsoid 
silhouette. 

A  secondary  source  of  clutter  prevalent  in  natural  scenes  is  the  contrast  produced 
by  shadows.  Shadows  have  traditionally  been  an  expensive  feature  to  add  to 
computer-generated  images.  Our  functional  model  approximates  shadows  econom¬ 
ically  by  projecting  the  tree  cluster  regions  onto  the  terrain  under  the  trees  and 
darkening  the  terrain  surface. 
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2.3  Obscuration 

In  addition  to  trees,  meteorological  clouds  and  man-made  smoke  screens  produce 
obscuration  which  can  adversely  effect  military  operations. 

We  model  clouds  in  a  manner  very  similar  to  that  in  which  we  model  trees,  using 
ellipsoids  with  functional  texturing  which  modulates  both  shading  intensity  and 
translucence.  There  are  two  significant  differences,  however.  First,  we  cluster 
trees  two-dimensionally  on  the  terrain  surface,  while  we  cluster  clouds  three- 
dimensionally  above  the  terrain.  Second,  our  cloud  model  is  dynamic,  allowing  the 
visualization  of  time-variations  and  their  effect  on  obscuration.  A  dynamic  model 
also  allows  rerunning  a  simulation  quickly  with  changes  in  initial  parameters  to 
study  a  variety  of  "what  if"  situations  effecting  decision  sensitivity  or  system  reli¬ 
ability. 

Our  dynamic  cloud  model  simulates  the  "parcel"  theory  of  cloud  formation,  which 
describes  cloud  formation  due  to  heated  bubbles  of  air  expanding  and  rising  into 
cooler  atmospheric  regions  forcing  condensation  of  moisture.  We  implement  this 
theory  by  computing  our  2-D  spectral  function  at  each  point  in  a  specified  cloud 
grid  defined  at  a  specified  altitude  (the  altitude  of  condensation).  The  function 
simulates  heat  energy  and  expands  the  cloud  ellipsoids  and  decreases  translucence 
to  represent  condensation. 

We  add  cloud  shadows  in  a  cost-effective  manner  by  projecting  the  cloud  function 
along  the  light  vector  and  darking  tree  and  terrain  surfaces  in  regions  where 
clouds  are  functionally  defined. 
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2.4  Variable  Resolution 


Despite  the  economy  of  our  model,  natural  scenes  can  be  so  complex  that  compu¬ 
tation  can  be  unacceptable  and  buffers  can  overflow  unless  efficient  data  manage¬ 
ment  techniques  are  applied.  Our  overall  approach  is  based  on  tailoring  the  data 
base  to  the  viewers  perception.  Thus  we  use  2-D  texturing  when  it  is  indistin¬ 
guishable  from  3-D  geometric  detail. 

We  have  extended  this  strategy  by  applying  hierarchical  data  structures, 
specifically  quadtrees,  to  represent  scene  features  in  varying  levels  of  detail  depen¬ 
dent  on  viewing  distance.  Quadtrees  provide  a  controlled  means  of  reducing  data 
volume  without  loss  of  significant  information.  We  apply  quadtrees  to  terrain, 
tree,  and  cloud  data  independently.  For  each  set  of  data,  we  define  a  minimum 
viewing  size  for  an  object  (a  triangular  polygon  for  terrain  and  an  ellipsoid  for 
trees  and  clouds).  The  minimum  size  is  used  to  compute  radial  sectors  within  the 
scene  for  a  given  eye  point,  with  the  sectors  centered  at  the  eye  point. 

Terrain  data  is  reduced  by  maintaining  the  original  grid  spacing  for  the  closest 
terrain  sector  and  successively  doubling  the  spacing  for  each  distant  sector. 
Because  doubling  the  spacing  decreases  sampling  resolution,  a  weighted  average  is 
used  to  filter  the  data.  Continuity  is  maintained  by  subdividing  a  small  percen¬ 
tage  of  the  nearest  polygons  in  each  distant  sector  to  match  the  farthest  polygons 
in  the  preceding  sector. 

Tree  and  cloud  data  are  reduced  by  a  similar  doubling  of  grid  spacing  accom¬ 
panied  by  a  doubling  of  ellipsoid  width.  This  is  equivalent  to  replacing  trees  in 
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the  foreground  with  groves  and  forests  in  the  distance. 

This  technique  is  very  effective  in  compressing  large  data  volumes  because  for 
each  successive  resolution  sector,  four  polygons  or  ellipsoids  are  replaced  by  one. 
This  is  important  not  only  for  computation  savings  but  also  because  it  allows  the 
modeling  of  large,  complex  scenes  with  limited  memory  buffers.  Without  such  a 
variable-resolution  data  management  scheme,  scene  models  would  be  limited  to 
either  small  regions  of  high  resolution  or  large  regions  of  low  resolution.  Defining 
resolution  as  a  function  of  view  point  tailors  the  imagery  to  perception,  so  that  no 
important  information  is  lost.  Finally,  we  have  implemented  parametric  control 
(minimum  viewing  size)  to  allow  selection  of  an  appropriate  tradeoff  between 
detail  and  computation  for  a  given  application. 

Figure  1  shows  an  example  of  a  3-D  scene  model  generated  entirely  from  func¬ 
tions.  The  use  of  texture  and  variable-resolution  geometric  data  allow  a  realistic 
representation  of  natural  complexity  and  clutter  necessary  for  an  effective  DETB. 

2.4  Enhancement  of  Real  Data 

A  great  deal  of  flexibility  is  provided  by  the  capability  to  generate  an  entire  scene 
background  under  functional  control.  For  certain  applications,  however,  it  will  be 
desirable  or  necessary  to  replicate  an  actual  geographic  region.  In  such  cases,  real 
data  can  be  used  and  enhanced  by  functional  modeling.  Figure  2  shows  a  model 
of  the  Fulda  Gap  in  Germany  using  real  terrain  elevation  data  and  real  feature 
data  specifying  treed  regions.  Functions  were  used  to  add  terrain  detail  and  color 
variations  and  to  insert  3-D  trees  within  the  specified  treed  regions.  Figure  3 
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shows  the  same  model  enhanced  with  clouds  and  cloud  shadows  to  provide  a  more 
realistic  simulation  of  natural  clutter  and  obscuration.  This  512-by-640  pixel 
image  took  under  10  minutes  to  generate  on  a  Silicon  Graphics  Personal  Iris 
workstation. 


3.  CONCLUSIONS 

Functional  modeling  combined  with  variable-resolution  management  is  a  cost- 
effective  means  of  providing  realistic  simulation  of  complex  natural  background 
scenes  for  Digital/Electronic  Terrain  Boards.  The  technique  provides  great  flexi¬ 
bility  for  generating  and  changing  scenes  which  include  realistic  clutter  and 
obscuration.  The  dynamic  cloud  model  can  be  used  to  simulate  varying  degrees  of 
obscuration  due  to  cloud  cover  and  smoke  screens,  as  shown  in  figures  4  and  5. 
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Figure  1.  Functional  Scene  Model 
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Figure  2.  Functional  Enhancement  of  Real  Data 


Figure  S.  Functional  Enhancement  of  Real  Data  with  Clouds  Added 
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ABSTRACT 

The  US  Army  Communications-Electronics  Command  (CECOM)  Center 
for  Night  Vision  and  Electro-Optics  (C2NVEO)  has  originated  a 
facility  for  Computer-generation  of  Realistic  Environments  with 
Atmospheres  for  Thermal  Imagery  with  Optics  and  Noise  (CREATION) . 
This  is  essentially  a  model  which  allows  the  user  to  synthesize  and 
analyze  infrared  images  of  target  and  backgrounds  with  atmospheric 
and  sensor  degradations.  Realistic  electro-optical  sensor  effects 
can  be  applied  selectively  to  both  synthetic  and  high  quality 
Forward  Looking  Infrared  (FLIR)  imagery. 

The  versatility  of  CREATION  is  a  valuable  tool  for  several 
applications,  such  as  generation  of  imagery  for  human  search, 
systems  analysis  and  the  visualization  of  important  image  proper¬ 
ties  that  are  pertinent  to  Automatic  Target  Recognizer  (ATR)  per¬ 
formance.  Enhancements  to  CREATION  are  planned  in  the  areas  of 
incorporating  fully  three  dimensional  targets  and  backgrounds,  and 
in  improving  the  ability  to  add  textures. 

INTRODUCTION 

The  important  differences  between  human  and  machine  visual 
performance  must  be  considered  when  imagery  is  chosen.  In  general, 
machine  vision  of  higher  resolution  than  human  vision  is  within  the 
present  state  of  the  art.  The  greater  repeatability  and  absence  of 
fatigue  are  also  important  assets  of  machine  vision.  Unfortunate¬ 
ly,  simple  perception  tasks  that  humans  perform  instinctively  are 
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beyond  present  machine  vision  capability,  and  image  artifacts  that 
are  disregarded  easily  by  humans  can  seriously  affect  ATR  perform¬ 
ance  . 

A  digital  electronic  terrain  board  is  being  established  at 
C2NVE0 .  The  CREATION  Facility  will  be  used  for  the  production  of 
synthetic  imagery  that  does  not  have  the  unavoidable  artifacts  of 
real  time  (30  frames/second)  dynamic  sequence  programming.  The 
trade-off  will  be  precision  at  the  expense  of  computation  time. 
This  report  outlines  C2NVEO  plans  to  pursue  using  CREATION  capabil¬ 
ities  for  producing  sets  of  imagery  for  human  observers  and  ATR 
performance  tests  on  first  and  second  generation  FLIR  imagery. 
Necessary  amendments  for  its  interface  with  the  terrain  board  are 
explored. 

The  CREATION  facility  currently  is  comprised  of  a  Hewlett- 
Packard  (HP)  series  900  model  320C  workstation  that  supports  only 
two-dimensional  graphics  and  has  only  7.5  megabytes  of  Random 
Access  Memory  (RAM) .  A  new  Silicon  Graphics  workstation  that 
performs  3D  operations  in  hardware,  and  has  the  software  for 
sophisticated  texture  merging,  is  being  procured  for  inclusion  in 
the  Facility.  Sample  imagery  is  being  generated  on  the  existing  HP 
configuration.  Mass  production  of  synthetic  imagery  has  been 
postponed  to  a  date  when  the  new  equipment  will  be  available. 
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I.  THE  CREATION  CAPABILITY 


A.  Geometry. 

The  version  of  CREATION  currently  being  used  at  C2NVE0  has 
insufficient  RAM  for  implementing  readily  available  rigorous 
three-dimensional  software.  Other  government  agencies  have  pro¬ 
cured  geometric  target  files  and  digitized  landscapes  of  particular 
geographic  locations  that  we  plan  to  interface  with  CREATION 
later.  Currently  specialized  RAM  economical  routines  that  were 
developed  in-house  are  being  implemented.  For  the  particular  case 
of  an  observation  point  with  limited  horizontal  motion  these  spe¬ 
cialized  routines  transform  perceived  shortcomings  into  advantages. 

1.  Landscape.  To  investigate  search  strategies,  a  wide 
horizontal  panoramic  view  and  narrow  vertical  field  are  required. 
Image  distortions  are  minimized  over  the  area  of  interest1  when  a 
polar  coordinate  geometry,  depicted  on  Simulation  1,  is  chosen. 

A  stationary  viewer,  at  the  center  of  the  coordinate 
system,  has  a  360-degree  search  field.  This  arrangement  is  also 
advantageous  for  helicopters  that  rise  for  a  short  time  over  tree 
lines.  Motion  is  simulated  by  shifting  rings  of  equal  viewer-to- 
object  distance  (range)  relative  to  each  other^ .  Simulation  2 
shows  a  horizontal  projection  of  Simulation  1.  This  landscape  has 
generic  features  that  would  not  occur  in  such  close  proximity  in 
nature . 
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Simulation  3a  shows  gray  shades  and  Simulation  3b  is  a 
range  integer  picture.  Each  pixel  has  a  one-byte  materials  parame¬ 
ter  that  is  converted  to  a  shade  of  gray  in  a  signature  prediction 
model,  and  a  one-byte  range  integer  that  is  a  look-up  table  address 
to  the  range  in  kilometers.  The  trees  in  Simulation  3  have  the 
same  range  integer  as  the  location  where  the  bottom  of  the  trunk 
was  inserted. 

2 .  Preparation  for  Insertions . 

a.  Natural  Objects.  The  bottom  half  of  the  image  in 
Simulation  2  shows  natural  objects  that  have  been  constructed  by  a 
semi-artistic  method.  A  large  branch  structure,  drawn  by  hand  with 
a  digitizer  pen,  was  reduced  and  distorted  to  form  a  set  of  varia¬ 
tions  on  the  input  shape  that  are  pieced  together  later  to  form  the 
complex  trees.  This  method  avoids  repeated  manual  drawings  and 
allows  selective  improvements;  desirable  combinations  of  features 
are  stored,  and  poor  choices  are  discarded.  The  final  results  are 
retouched,  if  necessary,  prior  to  scaling  and  application  of  dis¬ 
tortions  . 


Ground  textures  may  be  drawn  with  the  digitizer  pen, 
extracted  from  a  thermogram,  or  derived  analytically.  In  later 
sections,  the  realism  of  our  textures  will  be  demonstrated,  and 
limitations  of  the  flat  planes  approach  used  will  be  analyzed. 
Specialized  distortion  algorithms^  are  used  to  make  a  set  of  dif¬ 
ferent  trees  out  of  a  single  master,  and  a  gradual  change  in  dis¬ 
tortion  gives  a  two-dimensionally  constructed  tree  remarkable 
plasticity  in  planned  approach  sequences. 
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b.  Targets.  Rigorous  three-dimensional  targets  were 
tested  in  CREATION,  but  inputs  were  cumbersome  and  execution  was 
slow  when  what  were  known  to  be  sufficient  facets  were  input.  In 
parallel,  a  simple  target  geometry  (2  1/2D)  was  implemented  where 
textured  planes  are  shifted  relative  to  each  other.  Viewing  angles 
can  be  changed  only  to  a  certain  degree,  but  the  targets  do  not 
have  the  typical  artifacts  nor  the  cartoon-like  character  of  Com¬ 
puter  Aided  Design  (CAD)  constructions  which  contain  an  insuffi¬ 
cient  number  of  facets. 

A.  Modeling  and  its  Digital  Simulation  Implementation. 

1.  Thermal  Signatures.  Simulation  set  4  shows  the  fidelity 
of  synthetic  imagery  by  means  of  comparing  a  FLIR  image  to  a  syn¬ 
thetic  image  originated  with  CREATION  using  thermal  signatures 
predictions  generated  by  d'Agostino's  SPACE  model4.  This  model  is 
available  from  the  Xontech  Corporation.  Simulation  set  5  illus¬ 
trates  the  signature  changes  over  a  diurnal  cycle^.  The  SPACE 
model  would  require  excessive  RAM  if  implemented  concurrently  with 
CREATION  on  the  existing  HP  configuration;  therefore,  predictions 
were  performed  on  an  IBM  PC/AT,  and  data  files  ported  to  the  HP 
9000. 


The  limited  number  of  facets  used  in  the  validation 
exercise,  illustrated  on  Simulations  set  4  and  5,  would  have  been 
objectionably  cartoon-like  in  the  absence  of  texture  merging  and/or 
systems  degradation  simulations.  Successful  texture  merging  was 
achieved  by  a  distribution  of  materials  integers  that  are  assigned 
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to  the  same  output  from  the  SPACE  model.  The  SPACE  model  predicts 
the  mean  radiant  power.  Superimposed  are  the  fluctuations  that  are 
either  drawn  with  a  digitizer  pen  or  derived  analytically.  The 
standard  deviation  of  the  gray  shades  of  the  grass  blades  on  Simu¬ 
lation  4b  are  an  input  that  can  be  compared  to  fluctuations  on 
grass  areas  of  FLIR  imagery. 

2.  Atmospheres.  A  simplified  LOWTRAN  7®  equivalent  atmos- 

•  *7  . 

pheric  attenuation  routine,  called  LTR was  originated  by  G. 
Kornfeld  in  FORTRAN  on  the  IBM  4341.  It  explicitly  uses  equilibri¬ 
um  thermodynamics,  described  in  standard  textbooks®'®  to  eliminate 
LOWTRAN  6  look-up  tables.  Comparison  tests  performed  with  LOWTRAN 
6  show  less  than  1%  of  difference  for  all  cases  where  identical 
results  are  desirable  (comparable  agreement  with  LOWTRAN  7  is 
anticipated) .  Some  step  discontinuities  in  LOWTRAN  are  removed 
intentionally . 

A  specialized  regression  algorithm  is  used  for  very  large 
calculation  sets.  LOWTRAN  or  LTR  is  used  to  calculate  atmospheric 
transmissions  at  several  ranges.  These  pairs  are  inputs  to  the 
regression  algorithm  whose  output  are  two  parameters  that  are 
ported  to  the  HP.  More  details  are  given  in  a  later  section  where 
range  and  altitude  dependence  is  discussed. 

3.  Sensors.  The  Sensor  Degradation  Digital  Simulation 
Module  of  CREATION  is  probably  the  most  sophisticated  one  of  its 
type  currently  available.  The  reason  is  that,  unlike  most  other 
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similar  efforts,  our  module  is  based  on  methods  of  sensor  specifi¬ 
cations  that  include  nonlinearities,  asymmetry  of  spread  functions 
and  off  axis  resolution. 

Efforts,  by  other  agencies  of  which  we  are  aware,  use 
Fourier  Transformation  methods  that  are  advantageous  for  sensor 
performance  predictions  based  on  a  matched  filter  signal  to  noise 
ratio,  because  multiple  convolutions  can  be  replaced  by  products. 
Excellent  fast  Fourier  Transformation  software  is  readily  avail¬ 
able,  but  it  lacks  the  versatility  of  the  shift  variant  convolution 
method  used  in  CREATION.  Previous  publications  by  G.  Kornfeld1' 5 
illustrate  several  sensor  effects  that  would  be  prohibitively 
difficult  to  calculate  by  the  Fourier  Transform  method.  The  use  of 
CREATION  routines  for  visual  experimentation  and  the  on-going 
interaction  with  systems  analysis  efforts  will  be  the  subject  of  a 
later  section. 

II .  APPLICATIONS 

Two  human  observer  tasks  will  be  compared  to  typical  ATR 
simulations.  Important  differences  that  must  be  considered  are  as 
follows . 

a.  The  human  observer  can  distinguish,  at  most,  one  hundred 
shades  of  gray;  therefore,  results  of  a  thermal  signature  predic¬ 
tion  model  over  a  diurnal  cycle  cannot  be  displayed  with  a  unique 
normalization,  but  the  dynamic  range  of  12  bits  (4096  shades  of 
gray)  of  the  new  C2NVEO  2-color  FLIR  is  used  to  enable  uniform 
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scaling  of  radiant  powers  over  a  diurnal  cycle.  The  input  to  the 
ATR  is  a  matrix  that  cannot  be  visualized  by  humans,  but  normaliza¬ 
tion  can  be  consistent. 

b.  In  dynamic  sequences,  the  human  observer  integrates  over 
several  frames,  but  the  ATR  views  frames  individually.  The  ATR  is 
bothered  by  artifacts  within  a  single  frame  that  human  observers 
can  learn  to  disregard,  but  inconsistencies  between  frames  that 
cause  sudden  jumps  in  the  imagery  do  not  affect  ATR  performance, 
yet  can  be  very  bothersome  to  human  observers. 

c.  Machine  vision  can  be  performed  from  a  digitization  at 
the  focal  plane  of  the  sensor  after  preamplification,  but  prior  to 
display  degradation.  Digital  simulations  of  sensor  degradation  at 
the  focal  plane  will  be  compared  to  the  display  that  is  viewed  by 
the  human  observer. 

A.  Imagery  for  Enhancement  of  Current  Sensor  Performance  Predic¬ 
tions  . 

The  C2NVE01®  Static  Performance  Model,  used  for  the  prediction 
of  Thermal  Sensors,  and  the  Image  Intensifier  model  currently  are 
undergoing  extensive  revisions.  These  consist  of  model  unification 
and  conceptional  changes  to  include  impacts  of  new  inventions  and 
computer  code  revisions. 

The  use  of  CREATION  sensor  degradation  routines  play  a  major 
role  is  a  visual  experimental  series  performed  by  J.  Howe  and 
coworkers11.  Military  targets  were  extracted  from  close-up  FLIR 
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imagery,  and  selective  sensor  degradation  routines  were  applied  to 
these  FLIR  images  with  negligible  initial  degradation. 

Visual  experiments  that  currently  are  being  performed  address 
the  impacts  of  sampling  schemes  and  white  Gaussian  noise  addition. 
Some  problems  that  may  arise  during  further  developments  of  the 
experimental  series  are  outlined  here. 

1.  Assembly  Difficulties.  Current  visual  experiments  ana¬ 
lyze  the  impact  of  target  degradations  against  a  bland  background, 
but  future  insertions  into  FLIR  imagery  will  be  required  that  were 
not  necessarily  recorded  under  identical  conditions.  Objectionable 
artifacts  caused  by  mismatches  are  minimized  if  extreme  close  ups 
are  assembled,  then  extensive  size  reduction  is  performed. 

Synthetic  Imagery  Assembly.  Figure  1  illustrates  an 
unavoidable  assembly  difficulty  that  is  absent  in  the  synthetic 
imagery  of  CREATION,  but  present  in  the  specialized  application 
where  FLIR  images  are  assembled.  The  dashed  arc  depicts  the  bound¬ 
ary  between  a  target  of  gray  shade  GT  and  background  of  gray  shade 
GB.  In  digital  simulations,  boundary  pixels  are  assigned  the  gray 
shade  GT  and  a  weight,  w,  that  is  the  fraction  of  the  pixel  area 
that  belongs  to  the  target.  The  computer  generated  targets  and 
background  clutter  objects,  shown  on  the  bottom  half  of  Simulation 
2,  were  inserted  in  a  preliminary  background  of  gray  shades  zero  to 
allow  further  scaling.  Averaging  is  performed  over  non-zero  gray 
shades  and  the  pixels  of  gray  shade  zero  are  used  to  establish  the 
new  weights. 
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The  extractions  of 


targets  from  FLIR  images  were  performed  by  the  Systems  Analysis 
team.  Boundary  pixels  had  to  be  assigned  either  to  the  target  or 
background;  reconstruction  would  be  prohibitively  labor  intensive. 
The  heavy  outline  on  Figure  1  depicts  the  approximation  to  the 
dashed  arc.  Pixels  of  weight  that  exceed  0.5  are  assigned  to  the 
target,  but  their  shade  of  gray  have  target  and  background  contri¬ 
butions.  The  extracted  targets  are  inserted  into  preliminary 
backgrounds  of  gray  shade  zero  so  that  scaling  can  be  performed  by 
methods  described  in  the  previous  paragraph,  but  the  result  is  less 
precise  because  a  contribution  of  the  previous  background  has  bled 
in . 


In  current  visual  experiments,  targets  are  inserted  into 
a  bland  background  that  has  gray  shade  of  target  mean,  but  for 
later  experiments,  insertions  into  realistic  backgrounds  will  be 
required.  Merging  will  be  performed  according  to  weights,  but  the 
inherent  imprecision  of  the  extraction  method  can  nevertheless 
cause  objectionable  halos.  Two  assembly  method  compromises  are 
submitted: 

a)  Do  not  extract  the  precise  target,  but  surround 
it  with  part  of  the  previous  background.  The  mismatch  is  shifted 
from  the  region  of  importance  around  the  target  circumference  to 
some  irregular  shape  that  cannot  be  confused  with  a  logical 
boundary . 

b)  Use  weighted  insertions  similar  to  the  CREATION 
method  and  minimize  the  error  by  a  proper  choice  of  background. 
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12.  Non-Whits  Noise  Character.  Figure  2  shows  a  typical 
noise  spectrum  and  Simulations  set  6  shows  the  impact  of  white  and 
1/f  noise.  There  is  an  electronic  filter  that  consists  of  a  low 
frequency  cut-off  with  3  dB  point  of  frequency  f^  and  a  high 
frequency  roll-off  of  3  dB  point  fR.  At  a  knee  frequency,  fK' 
white  and  1/f  noises  have  the  same  power. 

(Simulations  6  a,b  demonstrate  the  C2NVE0  serial  scan 
calibrated  FLIR  where  32  detectors  are  time  delay  integrated  (TDI) ; 
a  digitized  single  frame  is  compared  to  frame  averaging  over  9 
frames.  Non-uniformities  do  not  exist  in  serial  scans,  thus  the 
streaks  are  caused  by  1/f  noise.  This  effect  is  apparent  on  the 
FLIR  monitor,  but  was  lost  in  the  reproduction,  thus  instead  digi¬ 
tal  simulation  is  shown. 

Two  distinct  noise  effects  are  present.  White  noise  is 
applied  to  individual  pixels;  and  1/f  noise,  caused  by  infrequently 
occurring  strong  voltage  spikes,  is  smeared  over  the  entire  pic¬ 
ture,  thus  acting  like  a  slowly  varying  non-uniformity  that  is 
reduced  by  frame  averaging.  A  non-uniformity  caused  by  different 
detector  characteristics  would  become  more,  not  less,  apparent 
after  frame  averaging. 

In  the  Static  Performance  Model1®,  the  noise  bandwidth, 
Afn,  is  calculated  by  combining  the  two  contributions,  although 
FLIR  image  la  clearly  shows  the  different  noise  characters.  The 
problem  is  solved  by  the  separation; 
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(1) 


Afn=  *W) 

where  afw  can  be  evaluated  numerically,  but 

is  a  rigorous  closed  form  solution.  An  important  difficulty  found 
in  the  numerical  solutions  of  the  static  performance  model  is  the 
approach  of  1/f  noise  to  infinity  when  f  approaches  zero.  The 
meaning  of  the  physics  involved  is  improved  and  computational 
methods  are  simplified  by  the  individual  treatment  of  the  noise 
sources . 


CREATION  accepts  the  two  noise  sources  as  equivalent 
temperature  differences  that  would  causes  the  same  voltage  differ¬ 
ence  as  standard  deviations  of  the  noises.  White  noise  can  be 
positive  or  negative  and  is  applied  to  all  individual  pixels;  1/f 
noise  consists  of  positive  and  definite  strong  voltage  spikes  that 
are  smeared  over  entire  lines  by  the  detector  motion. 

The  256  shades  of  gray  that  can  be  displayed  on  the 
monitor  are  correlated  to  the  scene  maximum  and  minimum  tempera¬ 
tures  that  are  used  to  scale  the  noise  standard  deviations.  Visual 
experiments  then  can  establish  their  impact  on  performance. 

B .  Search  Experiments . 

This  program  is  being  promoted  by  Mr.  W.  Hollis,  Deputy  Under¬ 
secretary  of  the  Army  for  Operations  Research. 
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A  cooperative  effort  is  planned  between  the  US  Army  Human 
Engineering  Laboratory  (HEL) ,  the  Institute  for  Defense  Analysis 
(IDA)  and  C2NVE0  for  the  investigation  of  search  strategies. 
C2NVE0  furnishes  undegraded  imagery  with  controlled  clutter  varia¬ 
tion;  IDA  and  HEL  perforin  visual  experiment  and  test  metrics  for 
clutter  description. 

Simulation  set  1  shows  the  plan  for  the  panoramic  views  in 
polar  coordinates  that  was  used  to  originate  four  landscapes  that 
simulate  different  geographic  locations.  One  of  these  was  chosen 
to  apply  controlled  increases  in  clutter.  Simulation  set  7  illus¬ 
trates  six  of  the  fifteen  clutter  steps  that  were  furnished  to  HEL. 
Similar  controlled  clutter  increases  also  will  be  applied  to  the 
other  landscapes  shown  on  Simulation  set  8. 

The  controlled  clutter  increase  was  performed  in  a  single  day 
because  CREATION  has  very  efficient  scaling  and  distortion  algo¬ 
rithms  and  performs  automatic  blending,  as  described  in  a  1988  SPIE 
presentation5.  The  telltale  halo  is  absent  because  each  tree  pixel 
has  its  shade  of  gray  and  a  weight  0<w<l  that  emerges  from  nonuni¬ 
form  scaling.  The  assembled  image  has  pixel  replacements  where  the 
fraction,  w,  is  the  insertion  gray  shade  and  the  fraction  (1-w)  is 
the  gray  shade  of  the  background.  All  pixels  where  w  >0.5  receive 
the  range  integer  of  the  insertion  point;  range  integers  where  the 
weight  of  the  insertion  w<0.5  remain  unaltered. 
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C .  Digital  Simulations  for  ATR  Tests . 

Two  important  considerations  that  impact  ATR  performance  will 
be  outlined  here. 

1.  Properties  of  the  Digitized  Focal  Plane  Image.  The  human 
observers  view  a  display,  but  the  ATR  is  furnished  with  a  digitized 
image  from  the  focal  plane,  after  preamplification  but  prior  to 
display  degradations.  Theoretically,  a  linear  response  from  the 
detectors  can  be  anticipated.  Digitization  that  is  finer  than  the 
noise  standard  deviation  would  be  counterproductive,  but  if  12 
bits/pixel  (4096  distinct  radiant  power  steps)  are  very  useful  for 
diurnal  cycle  analyses,  although  they  cannot  be  expressed  on  single 
frames . 

2.  Artifacts  of  tha  Scanning  System.  A  salient  feature  of 
CREATION  is  the  capability  to  precisely  simulate  the  properties  of 
different  scanning  strategies  and  their  impact  on  the  imagery  as 
seen  on  the  different  stages  of  the  sensor1. 

Simulation  set  9  illustrates  artifacts  caused  by  scanning 
strategies  that  the  human  observer  can  disregard,  but  might  seri¬ 
ously  hamper  ATR  performance.  Unfortunately  we  were  faced  with 
making  the  choice  between  gross  exaggerations  that  seem  unnatural 
or  simulation  of  effects  that  are  clearly  noticeable  on  the  HP 
monitor,  but  lost  in  the  reproduction. 

a.  The  False  Shadow  is  caused  by  a  rotating  mirror 
parallel  scan  system.  Simulation  9a  shows  the  impact  of  the  spread 
function  asymmetry  with  unidirectional  scan  that  gives  images 
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plasticity.  The  false  shadow  is  advantageous  to  the  human  eye,  but 
this  is  not  necessarily  true  for  the  ATR. 

b.  The  Zig-Zag  is  caused  by  a  very  scan  efficient 
bidirectional  scan.  Scan  efficiency  is  optimized  by  the  bidirec¬ 
tional  scan  with  mirror  motion  that  causes  alternate  pixel  strings 
to  be  read  from  left  to  right  and  right  to  left.  Resolution  is 
optimized  with  a  fixed  number  of  detectors  on  a  parallel  scan 
system  when  the  image  is  formed  by  interlacing  the  pixel  strings. 
This  optimization  neglects  the  asymmetry  of  the  spread  function 
that  prevents  perfect  registration  of  the  interlaced  scan  lines. 
Vercical  boundaries  appear  to  have  sawtoothed  characteristics.  The 
ATR  algorithms  that  compare  straight  and  jagged  edges  will  encoun¬ 
ter  problems  due  to  these  artifacts. 

c.  Recapitulation.  The  false  shadow  is  helpful  for 
human  perception;  the  zig-zag  is  not  significantly  bothersome  to 
humans,  but  was  the  cause  for  rejection  of  the  bidirectional  inter¬ 
laced  scan  for  ATR  systems.  Noninterlaced  bidirectional  scan  might 
cause  objectionable  flicker  and/or  widening  of  the  apparent  spread 
function  that  adversely  affects  human  perception.  The  ATR  analyzed 
single  frames,  thus  bidirectional  noninterlaced  scan  appears  as 
successive  unidirectional  scan  in  opposite  directions. 

Decisions  on  systems  design  must  be  based  on  a  compromise 
between  human  and  ATR  requirements,  where  the  difference  between 
valid  requests  and  "wishful  thinking"  by  the  ATR  manufacturers  must 
be  recognized.  Simulations  performed  with  the  specialized  CREATION 
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routines  that  address  artifacts  not  considered  in  Sensor  Perform¬ 
ance  prediction  programs  facilitate  these  decisions.  An  example 
where  CREATION  simulations  could  have  been  helpful  is  the  choice  of 
an  interlaced  bidirectional  scan  for  the  common  module  FLIR  based 
on  a  scan  efficiency  optimization.  The  systems  analysis  program 
does  not  consider  the  stabilization  effect  of  rotating  mirrors  or 
the  spread  function  asymmetry. 

III.  DIGITAL  ELECTRONIC  TERRAIN  BOARD  IMPLEMENTATION 

The  software  will  be  ported  from  the  HP  9000  to  the  Silicon 
Graphics  workstation.  In  parallel,  implementations  on  the  C2NVEO 
Autospec  super-minicomputer  will  be  investigated.  There  will  be  no 
severe  RAM  limitations,  therefore,  methods  are  now  feasible  that 
were  previously  discarded.  The  individual  parts  in  the  CREATION 
will  be  reexamined. 

A.  Geometry. 

1.  The  Landscape.  The  digital  electronic  terrain  board  must 
have  the  capability  to  process  DMA  data  and  accept  correlations  of 
feature  and  elevation  maps  performed  by  Faust  at  the  Georgia  Insti¬ 
tute  of  Technology1^.  For  precise  scenes,  enhancements  of  the 
feature  maps  that  give  logical  positions  for  natural  objects  will 
be  performed  in-house.  For  dynamic  sequences,  simplifications 
proposed  by  Boeing  will  be  analyzed. 

Cartesian  and  polar  coordinate  implementations.  Intui¬ 
tively,  it  may  seem  that  the  standard  three-dimensional  transforma¬ 
tions  of  DMA  data  are  less  prone  to  distortions  than  the  shifting 
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of  the  rings  of  equal  viewer  distance  used  in  CREATION.  In  reali¬ 
ty,  bad  distortions  and  objectionable  artifacts  can  and  do  occur 
when  limitations  of  the  standard  methods  are  misunderstood,  and 
rigorous  3D  polar  coordinates  will  also  be  implemented. 

Ms.  Lori  Scarlatos1^  explained  the  correction  of  distor¬ 
tions  of  mountain  crest  lines  caused  by  sampling  instead  of  inte¬ 
gration  of  input  DMA  data.  A  different  correction  is  used  in  the 
high  quality  flight  simulators  by  Evans  and  Sutherland.  Under 
certain  conditions  the  proposed  polar  coordinates  are  a  prevention 
of  distortions.  (Prevention  is  preferable  to  correction.) 

Each  method  has  its  advantages  and  disadvantages.  For 
stationary  observers,  or  observers  with  limited  horizontal  motion, 
prohibitively  dense  data  sets  would  be  required  to  approach  the 
undistorted  detailed  foreground  representation  of  CREATION.  Carte¬ 
sian  coordinates  should  be  chosen  for  flight  simulators  and/or  the 
view  from  a  moving  vehicle. 

Requiring  artistic  ability  for  users  to  generate  realis¬ 
tic  landscapes  must  be  avoided.  With  the  RAM  and  disk  space  avail¬ 
able  on  the  new  system,  this  will  be  achieved  by  pre-storage  of 
texture  planes  and/or  the  program  steps  that  create  them.  In 
addition  to  fractals,  the  simpler  texture  merging  used  in  the 
flight  simulator,  procured  by  the  US  Army  Environmental  Topographic 
Laboratory  (ETL)  from  Boeing,  will  be  investigated. 

Standard  b  splines  are  part  of  the  Silicon  Graphics 
software.  They  will  be  compared  to  the  distortion  algorithms 
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created  in-house  that  were  used  to  achieve  the  variety  of  the 
shapes  of  natural  objects  demonstrated  in  this  report.  The  versa¬ 
tility  and  automatization  of  the  C2NVE0  distortion  algorithms  was 
improved  and  only  RAM  limitations  prevent  current  implementations. 
Examples  are  as  follows. 

a.  Instead  of  range  integers  0<IR<256,  floating  point 
ranges  are  planned.  This  improvement  will  allow  us  to  show  fine 
distance  differences,  within  the  structure  of  three  dimensional 
trees,  that  are  important  for  radar  simulations. 

b.  The  set  of  distortion  parameters  was  increased. 

c.  In  addition  to  shade  of  gray  and  range  information, 
altitude  and  a  material  parameter  will  be  stored  for  each  pixel. 
This  capability  enhancement  will  allow  fully  automatic  merging  of 
many  trees  with  placement  and  scaling  governed  by  a  random  number 
generator.  The  range  dictates  scaling,  occlusions,  altitude, 
radiative  temperature;  and  the  material  parameter  defines  whether 
the  position  is  a  logical  insertion  position. 

2.  Preparation  for  Insertions.  Simple  three  dimensional 
transformations  use  the  same  preparations  as  for  insertion  of 
objects  in  the  Cartesian  and  polar  coordinate  geometry.  The  impor¬ 
tant  consideration  is  the  quality  of  the  computer  generation  of  the 
objects  that  must  be  inserted. 

a.  Natural  Objects.  The  trees,  assembled  from  parts 
drawn  with  the  digitizer  pen,  can  be  regarded  as  a  construction 
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from  deterministic  fractals  that  are  more  human  labor  intensive, 
but  less  computationally  intensive  than  the  standard  Mandelbrot  or 
Carpenter  fractals.  It  is  a  misconception  that  the  latter  require 
less  artistic  ability  than  the  origination  of  the  ones  for  CREATION 
because  random  number  operators  are  used  whose  results  then  are 
evaluated  by  the  user  for  their  structural  and  artistic  appropri¬ 
ateness  . 


Fractals  are  based  on  the  shape  repetition  in  nature 
that  is  very  apparent,  for  instance,  in  the  structure  of  a  cauli¬ 
flower,  which  is  built  out  of  progressively  smaller  dome  shapes. 
Similarly,  coast  lines  of  Sweden's  fjords,  the  little  canyons  that 
form  the  Grand  Canyon,  little  falls  on  the  sides  of  the  Niagara 
Falls,  or  cloud  formations  show  the  fractal  nature  that  is  computa¬ 
tionally  expressed  by  breaking  up  shapes  until  the  pixel  structure 
is  the  limiting  factor.  They  apply  to  some,  but  not  all  tree 
structures.  Also  the  manner  of  breaking  up  of  shapes  and  the 
scaling  of  the  random  numbers  require  considerable  insight  into  the 
structure  formations  as  well  as  artistic  ability. 

b.  Targets.  For  Computer  Aided  Design  'CAD),  a  Carte¬ 
sian  coordinate  system  with  the  origin  inside  the  target  is  advan¬ 
tageous.  Targets  are  built  out  of  facets  or  primitive  solids. 
Shading  algorithms,  which  are  readily  available,  apply  to  reflect¬ 
ed,  not  emitted,  radiation.  The  US  Army  Ballistic  Research  Labora¬ 
tory  ( BRL )  ,  in  cooperation  with  the  University  of  Utah,  has 
originated  a  very  sophisticated  geometric  model.  P.  Dietz^-4  points 
out  that  an  interface  attempt  with  the  PRISM  signature  prediction 
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model,  originated  by  the  Keweenaw  Research  Institute,  is  planned. 
In  parallel,  C2NVE0  is  investigating  an  interface  between  the  SPACE 
model  and  BRL  CAD  that  would  not  require  some  of  the  approximations 
necessary  for  the  PRISM  interface,  but  are  detrimental  to  BRL  CAD 
realism. 

C2NVE0  is  aware  of  other  agencies  who  have  con¬ 
structed  images  of  vehicles  using  rigorous  three-dimensional  CAD 
software  without  shading  algorithms.  These  might  be  readily  inter¬ 
faced  with  signature  prediction  models,  but  the  images  tend  to  be 
cartoon-like . 

B .  Modeling . 

1.  Thermal  Signatures.  An  interface  of  BRL  targets  with  the 
PRISM  model,  originated  by  the  Keweenaw  Research  Center,  is  funded 
by  the  US  Army  Tank-Automotive  Command  (TACOM) .  This  interface 
could  be  very  cost  effective  because  BRL  already  has  invested  many 
years  into  the  construction  of  the  geometry  of  many  targets. 

C2NVE0  has  procured  the  three-dimensional  target  signa¬ 
ture  prediction  model  from  the  Georgia  Institute  of  Technology, 
called  GTSIG15,  that  interfaces  with  their  geometric  model.  It  is 
less  sophisticated  than  the  BRL  CAD  geometry,  but  the  interface  is 
currently  operational,  not  a  future  plan. 

The  SPACE  model  must  also  be  implemented  in  parallel.  It 
is  simple  and  user  friendly,  and  after  some  limited  verifications 
it  compared  favorably  with  the  more  complicated  signature  predi c- 
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tion  models.  It  also  uses  primitive  shapes  not  only  facets  thus 
might  not  require  the  facet ization  of  BRL  CAD.  For  radar  models 
that  would  be  detrimentally  affected  by  false  corners,  facetized 
target  files  might  be  inappropriate.  The  interface  of  the  original 
BRL  CAD  with  SPACE  model  predictions  might  have  the  distinct  advan¬ 
tage  of  unifying  IR  and  radar  synthetic  imagery  creation  methods. 

2.  Atmospheres.  A  salient  feature  of  LTR  is  its  simplicity 
that  allows  transmission  calculations  to  individual  pixels  because 
a  specialized  curve  fit  algorithm  can  be  used  to  establish  parame¬ 
ters  for  a  range  and  altitude  dependent  simple  equation. 

=  exp{-(ffp)/[>+  <3> 

number  of  attenuators  in  a  slant  path 

average  attenuation  cross  section  calculated  from 
HIGHTRAN  or  Frauenhofer  lines 

are  empirical  outputs  from  the  regression  algorithm 

This  range,  R,  and  altitude,  A,  dependent  transmission  expression 
is  simple  enough  to  allow  calculations  to  individual  pixels  using: 
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Here,  Pout  and  Pin  are  output  and  input  radiant  powers  from  the 
Ix,  ly  location  on  the  picture  plane,  Paj_r  is  the  power  emitted 
from  the  ensemble  of  air  molecules  on  the  path  that  the  beam  tra¬ 
verses  . 


The  RAM  limitations  of  the  HP  9000  model  320C  forced  us 
to  calculate  the  parameters  of  Equation  (3)  separately  on  either 
the  IBM  4341  or  IBM  PC/AT.  Moreover,  transmission  is  only  impor¬ 
tant  when  a  long  range  approximation  is  justified. 

»  ex  p  j- r)  wUe*  cr^R  »  1  (5) 

The  approximation  ^  oc  R  was  made  because  RAM  limita¬ 
tions  forced  us  to  neglect  the  altitude  information.  Using  Equa¬ 
tion  (5)  instead  of  Equation  (3)  cuts  down  the  number  of  adjustable 
parameters  from  3  to  2,  but  has  the  disadvantage  that  Cf  is  empir- 

N 

ical,  but  <X  theoretical.  Broad  band  transmission  calculations  that 
do  not  make  the  Beer' s  law  approximation  of  exponential  decay  are 
made  with  only  two  input  parameters  that  are  easily  ported  to  the 
HP.  (The  normalization  of  (7  was  achieved  by  including  and 
a  conversion  from  p  to  R,  in  addition  to  the  units  conversion.) 

With  sufficient  RAM  at  our  disposal,  the  same  specialized 
curve  fit  algorithms  also  can  be  applied  to  LOWTRAN  7  results. 
There  are  some  theoretically  advantageous  properties  to  Equation 
(3)  because  CT  is  very  well  established,  and  temperature  and  pres¬ 
sure  dependencies  of  b  and  Ep  can  be  investigated. 


76 


3. 


Sensors.  No  drastic  changes  in  the  sensor  model  are 
anticipated.  This  module  will  be  an  ongoing  evolution  where 
properties  of  new  sensors  will  be  added. 

A  user-friendly  interface  that  was  developed  for  three 
different  audiences  is  already  operational  on  the  HP  9000  and  will 
be  ported  to  the  new  facility. 

a.  The  general  users  are  able  to  access  sample  atmos¬ 
pheric  and/or  sensor  degradation  options  via  a  code  number. 

b.  The  systems  analysts  are  able  to  input  the  same 
systems  parameters  as  for  the  Static  Performance  Model.  In  addi¬ 
tion,  they  have  the  option  to  access  the  specialized  routines  that 
make  CREATION  unique. 

c.  Software  developers  will  have  the  option  to  inter¬ 
face  new  CREATION  processes  in  a  modularized  manner  without 
corrupting  the  main  CREATION  architecture.  This  method  already  has 
been  used  by  the  CREATION  developers  for  several  projects,  but  to 
date  it  is  not  yet  user  friendly. 
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IV.  CONCLUSIONS 


CREATION  has  proven  its  usefulness  for  realistic  digital 
simulations  of  unusual  geometries  that  could  not  have  been 
developed  with  readily  available  software.  Its  usefulness  was 
demonstrated  for  realistic  digital  simulations  where  the  observa¬ 
tion  point  is  either  stationary  or  essentially  moves  vertically. 

The  polar  coordinate  geometry  can  be  regarded  as  a  "distortion 
prevention"  on  an  area  of  importance  where  targets  are  neither 
trivially  detectable  nor  too  far  away  for  acquisition.  The  presen¬ 
tations  on  corrections  of  distortions  that  were  attended  indicate 
the  attractive  features  of  prevention  instead  of  correction. 

Imagery  for  human  observer  performance  tests  was  produced. 
CREATION'S  versatility  in  controlled  clutter  positioning  and  auto¬ 
matic  scaling/distortions  enabled  the  production  of  imagery  for 
search  strategy  tests.  Precise  variations  of  sensor  effects  also 
helped  evaluate  the  impact  of  sensor  properties  on  human  observer 
performance . 

The  important  differences  between  human  observers  who  view  a 
display  and  the  ATR  that  receives  a  digitized  image  from  the  focal 
plane  of  a  sensor  prior  to  display  degradation  were  analyzed. 

CREATION  will  become  a  vital  part  of  the  digital  electronic 
terrain  board,  enabling  us  to  push  the  capability  of  this  state  of 
the  art  facility  to  its  productive  limits.  The  advantages  of  the 
specialized  methods  discussed  in  this  report,  achievements  of  other 
government  agencies  and  readily  available  visualization  methods 
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will  be  combined.  The  2  1/2-D  geometry  that  currently  is  imple¬ 
mented  on  the  HP  is  dictated  by  RAM  limitations.  Rigorous  3D 
algorithms  that  minimize  distortions  according  to  task  dependent 
rules  are  already  solved  on  paper.  Hardware  implementations  will 
be  used  whenever  possible  to  increase  speed;  but  for  specialized 
purposes,  rigorous  3D  algorithms  that  have  advantages  over  the 
generally  available  facetized  method  will  be  implemented  in 
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SIMULATION  9 

ARTIFACTS  CAUSED  BY  SCANNING 
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ABSTRACT 

This  paper  describes  a  method  for  the  computer  generation  of  visual  and 
infrared  scenes  and  sequences.  The  principal  use  of  this  program  has  been  for 
Automatic  Target  Recognition  and  ground  vehicle  vulnerability  studies  carried  out  at 
the  Grumman  Corporate  Research  Center  and  at  TACOM.  The  program  is  capable  of 
generating  realistic  visual  and  infrared  scenes  and  sequences  based  on  a  variety  of 
sources  including  topographic  maps,  DMA  maps,  and  photographs.  In  particular,  we 
have  developed  an  accurate  computer  model  of  the  Keweenaw  Research  Center's  standard 
scene  for  which  extensive  IR  measured  values  are  available.  If  specific  real  places 
are  not  desired  than  the  user  can  create  fictitious  scenes  using  natural  objects 
3uch  as  trees,  roads,  clouds  and  ground  vehicles  with  relative  ease.  The  program 
gives  the  user  complete  control  of  aspect,  orientation,  lighting  and  viewing  input, 
and  the  trajectories  of  all  moving  objects  in  the  scene  and  of  the  sensor  viewing 
the  scene.  Environmental  conditions  such  as  rain,  snow  and  fog  are  included  as  well 
as  seasonal  effects  in  both  the  IR  and  visual  scenes .  Effects  of  atmospheric 
transmission  is  also  included  using  a  late  version  of  LOWTRAN.  The  program  has  great 
generality  in  that  it  considers  both  faceted  and  quadric  objects  as  geometric 
primitives  and  therefore  can  use  the  available  data  bases  of  faceted  targets  as  well 
as  efficiently  generate  natural  objects  using  quadric  primitives.  Targets  are  simply 
considered  as  objects  in  the  scene  and  generated  together  with  the  background.  This 
avoids  the  problem  of  matching  targets  to  background  in  scale  and  intensity  which 
arises  when  the  target  is  "stuffed"  into  a  computer  generated  scene.  The  IR  model 
used  is  the  Inrared  Modeling  and  Analysis  (IRMA)  model  developed  for  the  USAF  by 
Grumman . 

The  paper  will  discuss  the  continuing  development  of  this  program  (under  the 
partial  sponsorship  of  TACOM) ,  strengths  and  limitations,  future  development  plans 
and  applications  of  the  program.  Extensive  examples  of  the  programs  capabilities 
will  be  given  via  videotapes . 
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INTRODUCTION 


The  use  of  synthetic  imagery  is  finding  a  widening  acceptance  as  a  tool  for 
analysis  in  many  tactical  systems  studies.  Automatic  Target  Recognition,  signature 
identification  and  the  inverse  problem  of  signature  suppression,  tactical  decision 
aids,  and  the  design  of  new  concept  vehicles  are  all  areas  where  synthetic  imagery 
can  be  used  to  an  advantage. 

For  the  past  several  years,  the  Grumman  Corporate  Research  Center  has  been 
engaged  in  a  research  program  to  develop  computer  software  which  will  produce  scenes 
and  sequences  of  tactical  imagery  which  are  both  visually  and  numerically  correct. 
Our  initial  efforts  in  the  area  of  Computer  Image  Gener-ation  (CIG)  were  concerned 
with  modeling  natural  phenomena  such  as  trees  and  clouds  in  a  computationally 
efficient  and  visually  realistic  manner.  This  was  achieved  by  using  quadrics  as  the 
basic  modeling  primitives  (Ref.  1).  However,  as  the  use  of  synthetic  imagery  grew  in 
the  military  community,  it  became  obvious  that  polygons  (faceted  models)  were  the 
favored  primitives  especially  with  respect  to  the  modelling  of  targets.  In  order  to 
make  the  growing  data  base  of  targets  available  to  us,  we  included  the  possibility 
of  mixing  both  polygons  and  quadrics .  This  further  gave  us  the  ability  to  match 
shapes  of  terrain  objects  which  was  somewhat  difficult  to  do  with  quadrics  alone. 
Geometrically,  the  program  now  contains  both  polygons  and  quadric  primitives  and 
each  is  used  to  the  best  possible  advantage  in  the  generation  of  an  arbitrary  scene. 

Our  initial  concentration  on  visual  scenes  has  now  shifted  to  infrared.  In 
conjunction  with  the  Keweenaw  Research  Center  (KRC) ,  we  initially  constructed  a 
visual  model  of  Standard  Scene  #  1  in  Houghton.  Then,  using  measured  data  collected 
at  Standard  Scene  #1  by  KRC,  we  "painted"  the  visual  scene  with  the  measured  data. 
Each  object  in  the  scene  was  given  gray  level  so  that  the  mean  and  standard 
deviation  of  the  gray  levels  over  that  object  was  in  agreement  with  the  mean  and 
standard  deviation  of  KRC's  measurements.  Using  this  procedure,  a  sequence  of  visual 
and  IR  scenes  was  constructed  showing  a  generic  tank  moving  across  the  Houghton 
scene  (Ref.  2) .  The  sequence  of  viewpoints  wa3  representative  of  the  views  which 
might  be  obtained  from  a  missile  seeker  while  the  target  was  under  attack.  This 
effort  demonstrated  the  general  usefulness  of  the  synthetic  imagery  and  specifically 
how  it  might  be  applied  in  a  target  recognition  problem. 

While  using  measured  data  certainly  gives  authenticity  to  the  IR  scene 
generation  program,  it  also  severely  limits  its  applicability  since  it  only  applies 
to  those  cases  in  which  the  measurements  are  available.  In  order  to  gain  the  full 
benefits  of  using  synthetic  imagery,  among  them  the  possibility  of  examining  scenes 
under  arbitrary  environmental  conditions,  a  phenomenology  modeling  approach  seemed 
necessary.  Consistent  with  this  goal,  we  added  an  IR  modeling  capability  to  the 
program.  The  model  we  chose  wa3  one  developed  by  Grumman  for  the  Air  Force.  The 
Infrared  Modelling  and  Analysis  program  (IRMA)  is  a  theoretical  model  which  uses 
empirically  extracted  thermodynamic  properties  for  prediction  of  both  terrain  and 
target  radiances  as  a  function  of  weather,  time  and  atmosphere  and  allows  for  the 
generation  of  IR  sequences  under  a  wide  number  of  input  conditions.  We  also  included 
in  the  program  the  latest  version  of  LOWTRAN  to  account  for  the  effects  of 
atmospheric  transmittance.  We  then  used  the  updated  program  to  produce  a  IR  sequence 
similar  to  the  one  described  above  as  well  as  many  single  IR  scenes  which  showed  the 
effects  of  weather  conditions,  time  of  day  and  a  variety  of  other  environmental 
conditions  (Ref.  3) . 

We  hi.ve  continued  to  gain  experience  in  the  use  and  usefulness  of  the 
synthetic  scene  generator  program  and  this  paper  includes  our  most  recent  improve¬ 
ments  and  thoughts  about  the  directions  which  we  intend  to  take  in  the  future.  Some 
of  this  thinking  is  influenced  by  the  fact  that  under  the  sponsorship  of  TACOM  we 
have  delivered  a  version  of  the  program  for  use  on  their  recently  acquired  CRAY. 

Much  of  our  thinking  about  ways  in  which  the  program  can  be  improved  and  new 
features  to  be  included  in  updates  to  the  program  is  and  will  be  influenced  by  the 
experience  of  the  TACOM  users.  Thus,  we  are  moving  the  program  from  what  had  been  a 
research  tool  to  what  will  be  a  user  tool. 
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The  topics  relative  to  new  features  and  new  directions  which  are  discussed  in 
detail  in  the  body  of  this  paper  are: 

We  are  interested  in  extending  the  scene  generation 
capability  from  a  specific  location  (Houghton)  to  arbitrary  loca¬ 
tion  and  therefore  have  been  included  the  capability  to  generate 
terrain  data  from  DMA  data  and  from  topographic  maps.  The  latter 
appears  to  have  greater  applicability  to  tactical  image  generation 

We  have  developed  a  terrain  following  technique  so  that 
ground  vehicles  following  specific  trajectories  in  the  scene 
will  follow  the  terrain  profiles  along  those  trajectories. 

Because  of  the  widespread  use  of  the  PRISM  model  in  the  user 
community  and  because  of  certain  limitations  of  the  IRMA  model,  we 
have  begun  discussions  with  KRC  on  including  PRISM  as  an  option 
for  generating  the  IR  radiances  for  the  scenes.  We  are  also 
considering  including  the  updated  version  of  the  IRMA  model  now 
under  development  at  Eglin  Air  Force  Base. 


Development  of  a  user  interface  which  will  allow 
inexperienced  users  to  develop  scenes  and  sequences  in  which 
various  objects  (terrain  and  natural  features  as  well  a3 
targets) ,  their  locations  and  motion,  and  the  associated  viewing 
parameters  can  be  arrived  at  by  the  viewer  in  a  "natural  way",  is 
receiving  a  great  deal  of  emphasis.  This  interface  is  the  most 
significant  step  in  increasing  the  value  of  the  program  to  the 
users . 

Terrain  Modeling 

Previously,  quadric  surfaces  were  the  main  modeling  primitive  in  the  CIG 
program.  These  surfaces  provide  great  flexibility  in  the  modeling  of  natural  objects 
such  as  trees  and  clouds  which  have  some  inherent  'fuzziness'  to  their  structure. 
They  are  not,  however,  as  well  suited  for  accurate  modeling  of  well-defined  objects. 
Representing  specific  terrain  features  such  a3  a  hill  with  a  second  order  surface 
presents  some  problems.  Given  measured  elevation  data,  one  must  calculate  some  sort 
of  'best  fit'  of  a  quadric  surface  to  that  data.  On  3ome  global  scale,  this  approach 
may  be  acceptable,  but  locally  the  real  world  irregularities  of  the  terrain  are  lost 
in  the  smoothness  of  the  approximating  quadric.  Some  of  these  irregularities  may  be 
of  importance  in  that  they  are  identifiable  features  for  that  specific  terrain. 

With  the  use  of  polygonal  modeling  primitives,  terrain  can  be  modeled  to  the 
accuracy  of  the  measured  elevation  data.  Any  irregularities  present  in  the  data  will 
be  present  in  the  model.  The  resolution  of  the  measured  data  thus  becomes  a  factor 
in  the  accurate  modelling  of  specific  geographic  regions.  Fig.  1  shows  an  image 
that  was  modeled  from  DMA  elevation  data  at  30  meter  postings.  The  grid  of  DMA 
elevations  was  triangularized  to  obtain  a  polygonal  model  that  could  be  viewed  in 
three  dimensions.  While  this  figure  shows  the  global  irregularities  of  the  region, 
any  features  that  have  dimensions  of  less  than  30  meters  may  fall  in  the  gaps  of  the 
measured  data  and  will  not  appear  in  the  model. 

In  order  to  model  local  features,  it  is  necessary  to  obtain  elevation  data  at 
a  resolution  in  accordance  with  the  features  of  interest.  Topographic  maps  can 
provide  this  data,  such  as  the  one  in  Fig.  2  showing  a  region  in  Houghton,  Michigan. 
Fig.  3  shows  that  same  region  where  elevation  increases  as  the  image  goes  from 
black  to  white.  Fig.  4  depicts  a  three  dimensional  view  of  the  polygonal  model  of 
the  hill  in  the  region.  This  model  was  obtained  by  digitizing  the  contour  line3 
from  the  topographic  map  and  then  fitting  triangular  polygons  to  those  contours.  The 
topographic  maps  used  to  model  this  area  also  contained  information  about  tree 
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lines,  individual  shrubs,  fences,  and  roads.  After  separately  modeling  these  types 
of  objects,  the  map  can  be  used  to  position  them  accurately  with  respect  to  the 
polygonal  terrain  model.  The  inclusion  of  these  types  of  objects  greatly  adds  to  the 
realism  and  ' recognizability'  of  specific  locations. 

Terrain  Following 

Terrain  following  allows  an  animator  to  describe  a  dynamic  object's  position 
and  orientation  over  a  terrain  surface  by  simply  entering  the  2-D  path  the  object 
should  follow. 

An  object's  position  and  orientation  in  the  3  dimensional  database  is 
described  by  6  parameters;  X,Y,Z, ROLL, PITCH, YAW.  These  parameters  must  be 
specified  for  each  frame  in  which  a  dynamic  object  is  present.  If  an  animator  is 
interested  in  sending  an  earth-bound  vehicle  (car, truck, tank, etc . )  over  a  three 
dimensional  object,  terrain  following  calculates  these  parameters  for  each  frame 
using  only  the  vehicle's  trajectory  start  and  end  points  and  velocity. 

To  produce  a  terrain  following  movie,  one  must  have  an  object  to  move,  a  path 
for  the  object  to  follow,  and,  of  course,  terrain.  Terrain  data  is  acquired  in  a 
variety  of  ways.  It  can  be  hand  digitized  from  an  elevation  map  (see  Fig.  2),  taken 
directly  from  Defense  Mapping  Agency  (DMA)  data,  or  created  with  an  object 
generation  program.  Any  object  compatible  with  the  renderer  can  be  sent  over  this 
terrain.  All  that  is  needed  is  a  text  file  defining  the  path  the  object  should 
take.  Currently,  this  file  consists  of  a  starting  position  (X,  Y)  and  a  number  of 
positions  toward  which  the  object  should  move,  along  with  the  velocity  it  should 
travel  (X,  Y,  VELOCITY) . 

Once  the  terrain  follower  has  the  required  information,  it  looks  at  the  first 
trajectory  and  calculates  the  number  of  frames  it  must  produce,  as  well  as  the  X  and 
Y  deviations  (DELTA  X,  DELTA  Y)  for  each  frame.  Additionally,  YAW  is  computed  at 
this  time.  For  each  frame  above,  the  terrain  follower  then  searches  the  terrain 
database  to  find  the  terrain  triangle  beneath  the  moving  objects  center  of  gravity. 
The  object's  elevation,  roll,  and  pitch  are  then  calculated  from  the  three 
dimensional  coordinates  of  the  triangles  vertices.  Time  spent  searching  the 
terrain  database  i3  reduced  by  keeping  track  of  both  the  current  terrain  triangle 
and  a  list  of  it's  neighbors.  If  the  new  object  position  is  not  encompassed  by  the 
current  terrain  triangle,  the  neighbors  list  is  examined.  If  the  correct  triangle 
is  not  found,  a  new  neighbors  list  is  constructed  by  rapidly  searching  the  terrain 
database.  Thi3  process  i3  repeated  for  each  path  data  point. 
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I  H£W _ Versions 

.  The  current  program  runs  in  the  Grumman  Corporate  Research  Center  on  a  Data 

|  General  MV10000  under  AOS/VS  and  a  CRAY  XMP  under  COS.  It  has  also  been  transported 

I  to  TACOM  to  run  on  a  CRAY  2  under  UNICOS  and  on  an  IRIS  4D  under  UNIX..  TACOM  has 

alos  run  a  VAX  version  under  VMS.  This  version,  while  not  as  fast  as  any  of  the 

I  above,  will  make  the  program  available  to  a  much  larger  group  of  users. 

For  the  most  part,  the  FORTRAN  77  source  code  is  easily  transportable  from 
machine  to  machine.  I/O  and  file  handling  are  the  major  areas  of  trouble.  There  were 
a  few  problems  in  this  area  in  that  the  format  used  at  Grumman  for  storing  image 
data  was  not  the  same  as  that  used  by  TACOM.  The  user  now  has  access  to  generic 
file  handling  routines  that  when  linked  with  the  program  allow  a  user  to  define  his 
own  format  for  image  storage. 

"Old"  code  such  as  IRMA  and  LOWTRAN  written  in  FORTRAN  IV 
are  not  guaranteed  to  move  as  easily  between  machines.  While  no  problems  have  yet 
been  encountered  with  the  IRMA  code,  LOWTRAN  uses  some  constants  that  are  too  large 
for  the  VAX  to  handle.  For  this  reason,  the  VAX  version  at  this  point  does  not 
include  LOWTRAN  for  atmospheric  effects.  Future  concentration  will  be  on  the 
"workstation"  versions  possibly  including  a  MAC  II  with  Transputers  which  will  allow 
parallelism  and  therefore  significant  running  time  reduction. 

Haex _ Interface 

The  synthetic  scenes  which  we  consider  generating  are  complex  with  respect  to 
the  number  and  kinds  of  objects  that  make  up  the  scene.  In  order  to  use  the 
i  generated  scenes  for  analyzing  the  adequacy  of  a  target  detection  algorithm  or  to 

I  measure  the  IR  characteristics  of  a  concept  vehicle,  the  scene  model  must  include 

vehicles  and  backgrounds  which  are  realistic  in  terms  of  clutter,  and  features  both 
natural  and  man-made.  Various  objects  within  the  scene  must  be  made  to  move  in  a 
realistic  manner  if  complete  sequences  are  to  be  examined.  The  first  component  of 
the  user  interface  is  a  tool  that  will  facilitate  the  creation  of  3uch  scenes, 
producing  the  databases  required  by  the  program  to  calculate  three  dimensional 
views.  The  second  component  of  the  interface  will  help  the  user  specify  object 
motions  throughout  a  sequence  as  well  as  the  motion  of  the  hypothetical  lens  through 
which  the  entire  sequence  is  to  be  viewed. 

Scene  building  is  the  term  we  use  for  the  first  component  of  the  user 
interface.  In  order  for  a  user  to  readily  put  together  a  scene,  various  tools  must 
be  available  that  will  simplify  the  process.  These  tools  are  being  designed  with 
some  sort  of  graphics  capabilities  to  enable  the  user  to  visualize  various  aspects 
of  the  scene  as  it  is  developed. 

The  first  part  of  building  a  scene  would  be  to  describe  the  terrain.  It  is 
necessary  that  this  happen  fir3t  so  that  objects,  such  as  trees,  defined  later  on 
can  be  positioned  on  the  terrain  (at  the  correct  elevation) .  The  terrain  could  be 
described  in  several  forms.  Files  containing  regular  (gridwork)  elevation  maps  could 
be  referenced  directly  whether  from  DMA  data  or  user  supplied.  A  more  powerful  tool 
for  creating  terrain  models  would  be  to  input  topographic  contour  lines  from  which 
the  interface  could  generate  a  polygonal  description. 

After  describing  the  terrain,  the  user  would  specify  the  remaining  contents  of 
the  scene.  In  addition  to  allowing  the  user  (if  they  so  desire)  to  specify  the 
exact  parameters  of  the  low-level  polygon  and  quadric  modeling  primitives,  generic 
objects  are  being  defined  that  will  enable  the  user  to  specify  at  a  high  level  the 
components  of  his  scene.  These  generic  objects  will  have  various  parameters  that  the 
user  can  modify  to  obtain  the  actual  object (s)  he  desires. 

An  example  of  this  type  of  generic  object  might  be  a  tree  line.  Rather  than 
having  the  user  locate  each  individual  tree,  he  can  input  parameters  that  would 
shape  a  generic  tree  line  to  the  scene  he  is  modeling.  Some  parameters  that  may  be 
used  would  be  the  coordinates  of  the  tree  line  as  well  as  the  average  tree  size. 


93 


shape,  and  color.  These  parameters,  when  combined  with  randomizing  factors,  would 
produce  a  tree  line  along  the  user  specified  path. 

One  more  example  of  a  generic  object  may  be  a  road.  The  parameters  involved 
here  would  be  a  description  of  the  centerline  of  the  road  and  an  associated  width. 
Other  possible  generic  objects  may  be  clouds,  lakes,  rivers,  and  individual  trees 
and  shrubs . 

Animation  of  the  objects  and  eyepoint  to  produce  a  sequence  of  images  is  the 
second  component  of  the  user  interface.  In  the  case  where  the  motions  are  predefined 
by  some  real  world  event,  the  user  will  simply  need  to  put  the  motions  in  the  format 
required  by  the  CIG  program.  An  example  of  this  would  be  the  simulation  of  a  sensor 
mounted  on  some  dynamic  platform  such  as  a  missile  or  an  aircraft  throughout  some 
predefined  flight.  The  dynamics  of  the  platform  will  provide  the  appropriate 
eyepoints  for  the  generation  of  the  sequence.  When  the  user  has  to  specify  the 
trajectories  of  objects  and  eyepoints,  once  again  some  tools  are  required. 

The  most  straightforward  of  the  trajectory  types  that  the  user  will  have  to 
define  is  that  of  a  vehicle  whose  motion  is  constrained  to  follow  the  terrain. 

In  this  case  the  user  must  provide  a  description  of  the  trajectory  for  only  two 
spatial  dimensions.  This  can  be  accomplished  using  key  points  and  interpolating 
splines  and  can  be  readily  visualized  in  a  two  dimensional  plot  of  the  region  of 
interest . 

For  objects  whose  motions  are  not  constrained  to  follow  terrain,  the  ele¬ 
vation  of  the  object  over  the  terrain  must  also  be  specified.  While  splines  in  three 
dimensions  can  be  used  to  solve  this  problem,  it  is  difficult  to  provide  the  user 
with  any  quick  three  dimensional  visual  feedback.  The  use  of  an  additional  2-D  plot 
of  elevation  versus  the  2-D  3pline  parameter  could  be  used  efficiently  to  display 
the  3-D  trajectory  to  the  user. 

While  the  ideal  interface  for  animating  sequence  would  make  use  of  real  time 
frame  generation,  we  must  do  the  best  we  can  given  the  limitations  of  hardware  and 
software.  Trajectories  defined  by  the  user  can  be  tested  by  low  resolution  sequences 
(both  spatially  and  temporally)  where  each  frame  contains  enough  content  to  check 
the  validity  of  the  defined  motion.  When  a  low  reso-lution  test  of  a  sequence  has 
shown  the  motion  to  be  what  the  user  wants,  a  final  high  resolution  version  of  the 
sequence  can  be  generated. 

Ua.as _ fiaXiaad, _ lestucea 

Within  the  SSG  program,  objects  are  textured  according  to  texture  models.  In 
the  generation  of  visual  images,  these  texture  models  are  used  to  vary  the  intensity 
of  color  across  an  object's  surface.  For  IR  imagery,  the  texture  models  are  actually 
used  to  distinguish  between  different  IR  object  classes  (trees,  road,  grass, etc.). 

A  texture  model  is  the  instantiation  of  a  texture  type  or  texture  function  with  some 
set  of  parameters.  Several  texture  models  based  upon  the  same  texture  type/f unction 
can  be  defined  at  the  same  time. 

A  texture  type  is  the  algorithmic  means  by  which  an  object  in  the  scene  is 
textured.  This  is  usually  a  mathematical  function  of  scene  coordinates  that  returns 
a  value  used  to  texture  (i.e.  modulate  intensity)  of  some  object.  There  are 
currently  three  predefined  texture  types  which  the  user  has  available.  Two  of  these 
are  based  on  summations  of  sines  used  for  the  texturing  of  natural  objects  like 
trees  and  ground,  while  the  third  type  varies  intensity  with  elevation  for  use  in 
generating  relief  map  images  of  terrain. User  defined  texture  types  are  now  supported 
by  the  SSG  program.  By  following  a  simple  interface  for  the  passing  of  parameters, 
the  user  can  provide  code  for  the  definition  of  any  texture  types  they  may  have 
developed. 

Another  extension  of  the  SSG  program  in  order  to  provide  the  user  with  a  more 
flexible  tool  is  the  concept  of  texture  functions.  Like  a  texture  type,  a  texture 
function  is  a  function  of  scene  coordinates.  Rather  than  returning  a  value  to  be 
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used  directly  in  applying  texture,  a  texture  function  returns  a  number  that 
identifies  a  texture  model.  The  texture  model  referenced  by  thi3  number  is  then  used 
to  texture  the  object. User  defined  texture  types  and  functions  can  be  instantiated 
as  models  in  the  same  way  as  the  predefined  types.  An  example  will  help  explain  the 
differences  between  texture  types,  models,  and  functions. 

Suppose  we  have  a  texture  type  that  generates  a  regular  pattern  of  circles. 

The  parameters  to  this  texture  type  might  specify  the  radius  of  circle  used  and  the 
pattern  spacing.  Two  distinct  texture  models  can  be  defined  as  instantiations  of 
this  texture  type.  Figures  1  &  2  shows  the  patterns  generated  using  these  models.  A 
texture  function  can  then  be  written  that  is  based  upon  a  checkerboard  tiling.  This 
function  will  return  a  one  or  a  two  depending  upon  the  location  within  the 
checkerboard.  A  third  texture  model  (Fig.  3)  can  be  defined  based  upon  this  texture 
function  by  supplying  specific  parameters  for  the  checkerboard  tile  size.  When  this 
model  is  referenced  to  texture  the  ground  at  some  (X,Y, Z)  coordinate,  the  SSG 
program  recognizes  it  to  be  based  on  a  texture  function.  The  result  of  that  texture 
function  is  either  a  one  or  a  two.  The  SSG  program  uses  texture  model  1  or  2 
accordingly  to  apply  the  correct  circular  pattern  at  the  original  (X,Y,  Z)  position. 
Figure  4  shows  the  resulting  pattern  generated  upon  the  ground.  Figures  5  and  6 
illustrate  the  use  of  these  capabilities  using  more  realistic  textures . 

It  should  be  noted  that  a  model  number  returned  by  a  texture  function  could 
reference  yet  another  texture  function  and  so  on.  This  allows  the  ambitious  user  to 
generate  extremely  complex  textures. 


gRQfiRAM _ REVISIONS 

Over  the  past  year,  some  attention  has  been  given  to  the  improvement  of  the 
scene  and  sequence  generator's  imaging  capabilities.  Previously,  antialiasing  (the 
removal  of  sampling  error  artifacts)  was  done  as  a  post  process  to  the  calculation 
of  the  visible  image.  However,  in  the  generation  of  IR  imagery,  aliasing  artifacts 
were  reintroduced.  The  conversion  from  visual  to  IR  required  taking  statistics  over 
image  segments  and  then  modulating  those  segments  to  match  the  predicted  statistics. 
The  segments  here  represented  object  classes  of  different  IR  characteristics .  Since 
each  pixel  in  the  image  could  belong  to  only  a  single  segment,  this  segmentation 
process  introduced  further  spatial  aliasing.  Furthermore,  since  the  modulation  from 
visual  to  IR  was  based  upon  statistics  gathered  on  a  scene  by  scene  basis,  temporal 
aliasing  could  occur  during  animations. 

The  current  approach  to  antialiasing  is  to  allow  pixels  to  contain  information 
from  more  than  one  object.  The  partial  contribution  of  an  object  to  a  pixel  is 
represented  as  a  pixel  fragment  with  attributes  such  as  color,  fragment  area,  and 
opacity.  When  all  the  fragments  for  a  pixel  have  been  determined,  they  can  be 
blended  together  to  produce  a  final  pixel  color.  This  method  produces  visual  images 
that  are  correctly  antialiased. 

The  aliasing  within  the  IR  imagery  (both  spatial  and  temporal)  was  a  result  of 
the  po3t  processing  which  assigned  each  pixel  in  the  image  to  a  segment  in  order  to 
obtain  statistical  information  for  the  object  class  represented  by  that  segment. 

This  aliasing  can  be  avoided  if  the  statistics  for  that  segment  within  the  visual 
image  are  known  a  priori.  The  modulation  from  visual  to  IR  can  then  be  done  at  a 
pixel  fragment  level  and  the  blending  of  those  fragments  will  produce  antialiased  IR 
imagery.  Since  the  intensity  variation  within  an  object  class  is  due  only  to  the 
texturing  applied  by  the  SSG  program,  knowing  the  statistical  nature  of  each  texture 
model  is  sufficient.  Thus  the  current  SSG  program  requires  that  the  mean  and 
standard  deviation  of  each  texture  model  be  specified  along  with  the  usual 
parameters.  This  method  eliminates  the  need  to  calculate  the  object  class  statistics 
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as  a  post  process  thus  saving  the  SSG  program  some  computational  overhead  in  the 
generation  of  IR  imagery. 

Data  Generation 

We  have  invested  a  portion  of  our  time  in  extending  the  Houghton  scene  in 
various  ways.  Most  of  the  geometric  modelling  was  done  on  a  Symbolics  3675  using  a 
software  package  called  SGeometry™.  This  program  enabled  us  to  extend  the  terrain 
database  to  encompass  a  wider  area.  We  were  then  able  to  model  additional  objects 
for  placement  within  this  area.  We  designed  some  components  for  an  airport 
consisting  of  a  tower,  runway,  hangars,  buildings,  and  a  light  plane.  A  car  and 
bridge  were  also  modeled  for  inclusion  in  the  scene.  Eventually,  scenes  containing 
objects  such  as  these  can  be  useful  in  high  value  target  recognition  studies.  Figure 
7  shows  a  view  of  the  plane  while  figure  8  displays  the  bridge. After  these  objects 
were  constructed  on  the  Symbolics,  the  associated  data  files  were  passed  to  our  host 
Data  General  MV10000  system  where  they  were  reformatted  for  use  by  the  SSG  program. 
It  should  be  stressed  that  this  modeling  is  geometrical  only.  It  is  our  intention  to 
model  these  objects  for  IR  simulation  in  the  near  future.  The  capability  to  quickly 
model  geometric  objects  has  played  an  important  part  in  our  extension  of  the 
Houghton  scene. 

We  have  also  developed  a  program  that  allows  a  user  to  generate  and  examine 
positional  data  that  may  be  required  by  the  SSG  program.  Thi3  program  runs  on  the 
MV10000  using  a  Gould  IP8500  for  image  and  graphics  display.  After  displaying  a  map 
of  the  terrain  as  an  image  on  the  Gould  and  identifying  the  mapping  from  image  to 
world  coordinates,  the  user  can  employ  graphic  overlays  to  interactively  define 
various  types  of  positional  data. 

The  first  of  these  data  types  is  called  a  region.  When  a  user  is  given  a 
terrain  database,  there  may  be  some  sort  of  accompanying  feature  database.  Features 
that  may  be  indicates  are  forest,  swamp,  field,  pond,  etc.  Working  with  this  feature 
information,  the  user  can  define  polygonal  boundaries  for  regions  representing 
specific  features.  The  polygonal  boundaries  of  these  regions  can  be  output  in  world 
coordinates  for  use  later  with  a  separate  program  that  will  3plit  the  original 
terrain  database  into  separate  pieces,  each  representing  a  single  region. These 
separate  pieces  can  then  be  textured  and  colored  appropriately  for  the  feature 
represented.  For  features  like  forests  however,  the  user  isn't  concerned  about  the 
region  a3  a  solid  area.  It  is  possible  instead  to  have  the  program  attempt  to  fill 
the  region  to  a  specified  density  with  randomly  positioned  discs  of  a  user  supplied 
radiu3 .  The  positions  of  these  discs  can  then  be  output  for  use  in  placing  trees 
upon  the  terrain. 

The  other  type  of  positional  data  generated  by  the  program  is  used  to  describe 
two  dimensional  trajectories.  The  user  first  specifies  control  points  to  define  a 
cubic  interpolating  spline.  These  control  points  can  then  be  edited  (moved,  deleted, 
etc.)  to  obtain  the  desired  trajectory  over  the  terrain  map.  Associated  with  each 
control  point  are  parameters  to  provide  local  control  of  the  tension,  continuity, 
and  bias  of  the  spline  (Ref.  5).  When  the  user  is  satisfied  with  the  defined 
trajectory,  the  (X,Z)  positions  (Y  is  elevation  in  the  SSG  system)  along  the  spline 
can  be  output.  These  positions  can  then  be  used  to  define  trajectories  for  animation 
of  objects  or  eyepoints  within  a  sequence. 


Terrain-Object  Handling 

Given  the  position  data  as  generated  by  the  program  discussed  above,  the  user 
might  want  to  place  a  tree  at  all  the  positions  that  filled  a  region,  or  calculate 
the  location  and  orientation  of  a  vehicle  as  it  moved  along  some  trajectory. 
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Originally  there  existed  a  program  for  each  of  these  tasks,  one  to  place  trees  and 
one  to  determine  vehicle  motion.  Rather  than  stamping  out  carbon  copies  of  some 
predefined  tree,  the  tree  placing  program  had  some  randomness  for  parameters  such  as 
height,  width,  and  color.  This  produced  a  forest  of  trees  distributed  around  some 
generic  tree  model.  This  approach  works  well  until  one  wishes  to  place  something 
other  than  trees  on  the  terrain.  Making  a  copy  of  the  code  and  patching  it  to 
produce  the  new  object  didn't  seem  to  be  a  very  practical  solution  since  that  leads 
to  N  programs  to  place  N  different  objects  on  terrain  where  the  bulk  of  those 
programs  i3  identical. 

Our  solution  was  to  develop  a  single  program  that  placed  objects  on  terrain 
using  user  defined  object  templates.  The  program  takes  as  input  a  polygonal  terrain, 
a  list  of  positions,  and  an  object  template.  For  each  (X,Z)  position  it  calculates 
Y,  the  elevation  of  the  terrain  as  well  as  the  attitude  of  the  underlying  polygon  at 
that  point.  These  values  are  used  to  generate  an  instance  of  the  object  template  at 
the  current  position.  Early  on  in  the  development  of  this  program  it  was  seen  that 
the  user  would  need  access  from  within  an  object  template  to  some  of  the  variables 
calculated  internally  by  the  program.  At  the  very  least,  the  user  would  need  to 
refer  to  the  (X,Y, Z)  position.  Elementary  arithmetic  capabilities  as  well  as  some 
means  of  specifying  randomness  were  also  needed.  These  concerns  indicated  that  a 
parser  should  written  in  order  to  interpret  the  object  template.  During  the  output 
phase  of  the  program,  the  parser  would  substitute  values  of  the  program's  internal 
variables  for  symbols  within  the  object  template  and  perform  any  arithmetic 
necessary. 

Once  it  was  determined  that  a  full-fledged  parser  needed  to  be  written,  we 
were  able  to  provide  the  user  with  capabilities  above  and  beyond  the  ones  mentioned 
above.  All  of  the  standard  arithmetic  operations  are  supported 

Parentheses  can  be  used  to  change  the  order  of  operations  within  an  expression.  The 
user  now  has  access  to  the  random  functions  N(x£,B)  and  U (min, max)  for  generating 
normal  and  uniform  distributed  values.  Intrinsic  functions  such  as  MIN,  MAX  and  ABS 
also  exist.  Along  with  the  set  of  predefined  symbols  (X, Y, Z, ROLL, PITCH, YAW, etc . ) 
that  define  the  state  of  the  terrain  at  the  current  position,  the  user  can  also 
define  his  own  symbols.  As  the  program  and  parser  were  used,  it  became  apparent  that 
file  acce33  and  conditional  output  would  be  useful.  The  parser  was  then  extended  to 
include  basic  file  handling  routines  and  an  IF-THEN-ELSE  capability. 

An  example  object  template  for  the  definition  of  a  tree  is  shown  in  Fig  9.  It 
can  be  used  to  generate  quadric  objects  to  represent  trees  whose  color  is  normally 
distributed  about  the  RGB  value  (50,180,90).  The  radii  of  the  quadrics  are  also 
normally  distributed.  The  Y  radius  is  assigned  to  the  symbol  HT  for  later  use  in 
positioning  the  center  of  the  quadrics  above  the  terrain.  This  example  demonstrates 
the  initial  use  of  templates  to  produce  a  data  file  containing  quadric  descriptions 
representing  trees  placed  on  terrain.  We  have  also  used  this  template  approach  to 
produce  data  files  describing  the  motion  of  a  vehicle  as  it  follows  a  trajectory 
across  the  the  terrain  as  well  as  eyepoint  motions  along  another  trajectory  while 
tracking  that  vehicle. Thus  the  simple  object  placing  program  has  developed  into  a 
powerful  tool  for  handling  terrain  and  objects. 


Comment:  RGB  color 

N(50, 5)  N(180, 15)  N(90,7) 

Comment :  Quadric  Radii 
N(8,l)  HT-N(60,5)  N(8, 1) 

Comment:  Position  of  center  of  quadric 
X  Y+U (0, 0 . 3) *HT  Z 
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We  have  discussed  and  demonstrated  some  of  the  newly  developed  capabilities  of 
the  Grumman  visual  and  IR  scene  generation  program.  We  believe  that  these  changes, 
some  of  which  have  already  been  made  and  some  of  which  are  anticipated,  will  make 
our  program  a  more  useful  tool  in  the  analysis  of  many  tactical  situations.  Further, 
we  hope  that  some  of  these  changes  will  eventually  make  the  program  available  to  a 
larger  number  of  users  and  that  their  experiences  will  in  turn  feedback  to  us  in  a 
way  that  will  cause  the  program  to  be  improved  and  made  more  useful. 
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ABSTRACT 

The  TACOM  Thermal  Image  Model  (TOM)  is  an  interactive  computer 
code  which  simulates  the  performance  of  IR  imaging  sensors  operating  in 
realistic  battlefield  environments.  Its  ability  to  produce  simulated  imagery  has 
proven  useful  in  applications  including  sensor  and  tracking  algorithm 
simulations,  obscurant  model  development,  and  vehicle  design  evaluations. 

This  paper  reviews  die  development  and  current  functionality  of  TOM, 
concentrating  on  descriptions  of  the  most  recent  distribution  version  (3.0)  and 
two  new  modules  which  treat  staring  Focal  Plane  Array  (FPA)  and  Image 
Intensifier  sensors,  respectively. 

1.0  INTRODUCTION  AND  BACKGROUND 

The  TACOM  Thermal  Image  Model  (TOM)  development  was  initiated  to  address  the  problem  of  evaluating  the 
infrared  signature  vulnerability  (probability  of  detection)  for  individual  concept  vehicle  designs.  Standard  analytic  methods1 

! 

evaluate  target  probability  of  detection  on  the  basis  of  average  signature  and  target  size.  While  this  approach  provides  a  very 
convenient  way  to  translate  imaging  sensor  capabilities  into  detection  range  performance  measures,  it  provides  no 
sensitivity  to  specific  vehicle  shapes  and  signature  patterns  which  may  form  cue  features  increasing  the  probability  of 
detection  and  recognition. 


The  TOM  approach  to  the  problem  is  to  start  with  target  imagery,  impose  natural  and  battlefield  atmospheric 
effects,  treat  the  degradation  caused  by  thermal  imaging  sensors,  and  provide  imagery  for  evaluation  by  techniques  including 


J.A.  Ratches  et  al..  Night  Vision  Laboratory  Static  Performance  Model  for  Thermal  Viewing  Systems.  ECOM- 
7043,  U.S.  Army  Night  Vision  and  Electro-Optics  Laboratory,  April  1975. 
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human  interpretation,  statistical  analysis,  and  tracking  algorithms.  In  this  way,  all  of  the  target  infrared  signature 
characteristics  important  for  any  particular  application  are  inherently  considered. 

Development  of  TTIM  was  started  in  1984;  a  version  including  the  LOWTRAN6  natural  atmospheric  model,  the 
AMGREN/ACT II  smoke  model,  and  a  scanning  array  IR  sensor  model  was  delivered  to  TACOM  late  that  year.  By  1987, 
the  model  had  matured  considerably  and  it  was  decided  that  TTIM  might  be  of  interest  to  the  modeling  community  at  large. 
In  October  1987,  the  first  TACOM  Thermal  Image  Model  users  meeting  was  held  to  release  version  2.3  of  the  model  to  the 
DoD  community.  In  April  1988,  a  second  meeting  was  held  to  release  the  model  contractors.  Version  3.0  was  released  to 
users  in  March  1989.  As  of  this  writing  there  are  28  registered  TTIM  users. 

There  are  a  number  of  references  (including^  .4 ,5)  which  describe  prior  TTIM  developments.  The  reader  is 
encouraged  to  refer  to  those  for  details  concerning  the  basic  modeling  techniques.  Our  goals  in  this  paper  are  to  describe  the 
modeling  philosophy  behind  the  latest  TTIM  release  (version  3.0),  present  two  new  sensor  models  which  are  undergoing 
validation  and  preparation  for  release  later  this  year,  and  provide  an  example  illustrating  simulation  experiments  encouraged 
by  the  TTIM  approach. 

2.0  TTIM  VERSION  3.0 

The  structure  of  the  latest  TTIM  version  is  depicted  in  Figure  1.  The  capabilities  emphasized  in  the  current  release 
include  a  more  generalized  vertical  structure,  more  formalized  definitions  of  module  interfaces  for  improved  horizontal 


B.K.  Matise  et  al.,  "Simulating  the  Performance  of  Imaging  Sensors  for  Use  in  Realistic  Tactical  Environments", 
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B.K.  Matise  et  al.,  "TACOM  Imaging  Sensor  Simulation  Model  for  Vehicle  Signature  Vulnerability  Evaluation", 
Proc.  of  the  Sixth  Annual  KRC  Symposium  on  Ground  Vehicle  Signatures.  August  1984. 

TJ  Rogne  et  al..  Development  of  an  Imaging  Sensor  Simulation  Model.  OMI-118(I),  OptiMetrics,  Inc., 
November  1984. 


C.S  Hall  et  al.,  TACOM  Thermal  Image  Model  Technical  Reference  and  User's  Guide.  OMI-183,  OptiMetrics. 
Inc.,  December  1986. 


102 


interchangeability,  and  the  definition  of  a  core  set  of  parameters  which  allow  modules  to  more  easily  share  a  common 
situation  reference. 


The  vertical  structure  divides  the  model  into  eight  functional  areas  representing  the  components  of  most 
engagement  modeling  problems.  Summarizing  each  area,  target  and  sensor  geometries  are  defined  by  a  scenario.  The 
engagement  takes  place  in  the  context  of  a  scene  representing  the  appropriate  target  and  background  signatures  viewed  from 
the  proper  sensor  perspective.  Natural  and  battlefield  effects  modify  how  the  scene  appears  to  the  sensor.  The  sensor 
measures  apparent  scene  properties  limited  by  its  design  capabilities.  The  sensor  output  may  be  processed  before  it  is 
displayed  or  acted  upon.  Finally,  the  auto-sequence  function  is  a  generic  name  for  the  functions  required  when  a  simulation 
is  dynamic.  For  example,  in  a  missile  tracker  simulation,  the  auto-sequence  function  could  include  a  flight  dynamics  model 
allowing  closed-loop  flight  simulations. 

The  full  benefit  of  a  general  vertical  structure  is  not  achieved  without  a  modular  approach  to  implementing  each 
function.  The  advantages  include  a  common  structure  that  can  be  applied  to  most  engagement  problems,  only  requiring 
changes  in  modules  specific  to  the  application.  Through  consistent  application,  a  library  of  modules  is  envisioned  that  can 
be  mixed  and  matched  to  meet  specific  needs.  The  two  smoke  modules  are  a  relevant  example.  The  AMGREN/ACT  II 
model  treats  aspects  of  the  smoke  transport  and  radiative  transfer  problems  in  fine  detail  at  great  computational  expense;  it 
is  general  enough  to  undertake  completely  arbitrary  smoke  system  design.  In  some  applications,  the  detail  was  not  needed 
or  desired;  a  fast  running  but  accurate  model  capable  of  representing  typical  inventory  smoke  systems  was  required.  This 
resulted  in  the  development  of  a  version  of  EO-SAEL  COMBIC®  as  a  TOM  module.  Now,  both  modules  are  available  to 
support  future  applications  at  the  discretion  of  the  analyst 


D.W.  Hoock  et  aL.  Combined  Obscuration  Model  for  Battlefield-Induced  Contaminants  COMBIC.  TR-0221-1 1, 
U.S.  Army  Laboratory  Command  Atmospheric  Sciences  Laboratory,  October  1987. 


Modularity  also  protects  the  future  utility  of  the  code.  The  state-of-the-art  in  several  simulation  functional  areas 
such  as  scene  modeling  is  constantly  changing.  It  is  intended  that  TOM  provide  a  framework  to  combine  the  best  and  most 
appropriate  functional  models  at  any  given  time  to  form  a  complete  through-the-sensor  simulation  package. 

The  final  major  refinement  in  version  3.0  is  the  concept  of  core  simulation  variables.  There  is  an  obvious  need  in 
simulation  to  make  sure  that  all  models  are  working  from  common  assumptions.  For  example,  the  weather  conditions  used 
by  one  model  to  predict  signatures  ought  to  match  the  conditions  used  to  predict  smoke  transport  and  atmospheric 
transmittance.  To  date,  this  has  been  the  responsibility  of  the  analyst  Now,  there  is  a  mechanism  to  ensure  this 
automatically. 

TOM  version  3.0  was  distributed  to  users  during  March  1989.  It  should  be  mentioned  that  version  3.0  has 
removed  all  known  extensions  to  FORTRAN  77  used  in  prior  versions.  Along  with  a  new,  flexible  display  driver,  this 
will  facilitate  transport  of  the  code  to  workstation  environments.  The  image  intensifier  and  FPA  sensor  models  discussed  in 
the  next  sections  are  planned  for  release  later  this  year. 

3.0  IMAGE  INTENSIFIER  MODEL 

Although  TOM  was  originally  developed  for  use  in  the  infrared  spectral  region,  the  atmospheric  models  retain 
their  capability  to  operate  in  the  visible  and  near-IR  bands.  In  1987,  the  development  of  an  image  intensifier  sensor  model 
was  initiated  by  TACOM  resulting  in  a  stand-alone  model7.  With  the  completion  of  TOM  version  3.0,  the  image 
intensifier  model  is  currently  being  validated  and  prepared  as  a  module  for  release  to  users  later  this  year. 

The  structure  of  the  image  intensifier  model  is  shown  as  Figure  2.  Its  current  form  maps  into  three  TOM  modules 
representing  scene  model,  sensor  model,  and  display  functions.  The  sensor  model  uses  a  completely  linear  system 

7-  TJ.  Rogne,  An  Image  Intensifier  Simulation  Model  -  Program  IIMQDEL.  OMI-209,  OptiMetrics,  Inc.,  June 

1987. 
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representation  for  image  intensifier  or  direct  view  optics  devices.  The  model  includes  three  noise  sources  (representing 
background,  tube,  and  eye  sensitivity  limits)  and  three  classes  of  transfer  functions  (representing  resolution  limits  of  optics, 
tube,  and  the  observer’s  eye).  For  convenience,  all  of  the  transfer  functions  are  applied  in  the  frequency  domain.  The  order 
for  application  of  noise  sources  and  transfer  functions  is  shown  in  the  figure. 


The  model  calculations  begin  with  computation  of  the  mean  tube  signal  current  produced  by  each  scene  pixel: 


1  = 


ref  t0PH  PW 


4F . 


|  J l(X)RT(X)  dXxlO  13 


AX 


where 


I 

3 

pixel  photocurrent  (A) 

f 

3 

objective  focal  length  (cm) 

F# 

3 

objective  f/# 

*o 

3 

objective  transmittance 

PH 

= 

scene  pixel  height  (mrad) 

PW 

= 

scene  pixel  width  (mrad) 

P 

= 

scene  pixel  diffuse  reflectance 

Itt) 

= 

source  spectral  illumination  (W/m2-pm) 

R-iA) 

3 

tube  responsivity  (mA/W) 

The  model  includes  illumination  spectra  for  sun,  moon,  and  clear  night  sky  conditions,  as  well  as  artificial  illumination  as 
provided  by  a  2870K  standard  tungsten  lamp.  The  signal  current  is  converted  to  the  number  of  photo-electrons  produced 
during  one  eye  integration  time  by  the  expression: 


where  Ne  =  #  photoelectrons/pixel/eye  integration  time 

te  =  eye  integration  time(s) 
e  -  election  charge  (Q 

Noise  in  the  tube  signal  and  dark  current  are  governed  by  Poisson  statistics  so  that  for  each  component,  the  noise 
variance  is  equal  to  the  mean  election  flux.  An  empirical  eye  sensitivity  model^  expressed  as  a  noise  source  is  included. 
The  model  represents  the  fact  that  at  low  light  levels,  the  eye  sensitivity  can  be  limited  by  photon  rate  statistics,  but  that  at 
higher  light  levels  the  sensitivity  reaches  a  limit  defined  by  contrast 

Transfer  functions  used  in  the  model  are  shown  in  Table  1.  Generic  functional  forms  are  available  for  the  optics 
and  eye  [8,9];  tabular  input  must  be  used  to  define  the  tube  resolution. 

Sample  imagery  produced  by  the  model  is  shown  in  Figure  3  based  on  a  night  goggle  specification  provided  in 
Table  2  and  a  scene  reflectance  map  (original  image)  derived  from  visible  imagery.  The  images  illustrate  the  influence  of 
light  level  on  this  postulated  sensor's  ability  to  represent  the  original  scene.  The  reader  should  be  cautioned  that  since  the 
original  scene  reflectance  map  was  derived  from  visible  imagery,  it  does  not  accurately  represent  the  specific  near-IR 
reflectance  of  each  scene  component  (particularly  the  vegetation).  Thus,  no  specific  performance  conclusions  should  be 
drawn.  Nevertheless,  the  example  demonstrates  the  model  capability. 


JJE.  Rice,  Documentation  and  Technical  User's  Guide  for  the  Night  Vision  Laboratory  Image  Intensifier  Computer 
Code.  OMI-82-024,  OptiMetrics,  Inc.,  December  1982. 

Robert  Ji.  Corriveau,  The  Image  Intensifier  Model.  Unpublished  report  of  work  at  the  U.S.  Army  Night  Vision 
and  Electro-Optics  Laboratory,  June  1978. 
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4.0  FOCAL  PLANE  ARRAY  SENSOR  MODEL 

As  part  of  development  of  the  Missile  Enhanced  TACOM  Thermal  Image  Model  (MET -TIM) 1,  a  version  of 
the  original  scanning  ER  sensor  model  was  modified  to  treat  Focal  Plane  Array  (FPA)  sensors  with  specific  emphasis  on 
PtSi  detectors.  Major  changes  to  create  the  FPA  model  included  the  addition  of  a  responsivity  model,  a  simulation  of  fixed 
pattern  noise,  the  addition  of  new  transfer  functions  representing  CCD  effects,  and  the  addition  of  new  noise  terms. 

The  responsivity  model  for  Schottkey  barrier  type  detectors  predicts  the  spectral  response  as  a  function  of  emission 
factor  and  Schottkey  barrier  height: 

(  x<b  "l2 

r(X.)  =  C  1-  —7-” 
iv  he  ; 

where  <t>b  =  Schottkey  barrier  height  (eV) 

X.  =  wavelength  in  microns 

Ci  =  emission  factor  (eV'1) 

For  a  PtSi  detector,  representative  values  for  the  the  barrier  height  and  emission  factors  are  0.23  eV  and  0.33  eV"1, 
respectively. 


A  spatial-domain  sampling  model  and  a  fixed  pattern  noise  simulation  were  derived  to  treat  fill  factor  and 
uniformity  problems  associated  with  FPAs.  The  sampling  model  explicitly  represents  active  detector  areas  in  the  array 
pattern.  The  model  is  necessary  to  account  for  energy  loss  and  aliasing  effects  in  present  arrays  which  have  fill  factors 
varying  from  10  to  100%. 


10-  R.  Freeling  et  al.. 


lids,  OMI-302,  OptiMetrics,  Inc.,  September  1988. 


1  R.  Freeling  et  al.,  "Update  on  Recent  Improvements  to  the  TACOM  Thermal  Image  Model  (TTIM)”,  Proc.  of  the 

August  1988. 
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At  the  present  time,  fixed  pattern  noise  is  represented  as  a  random  variation  in  responsivity  across  the  array, 
uncorrelated  from  detector  to  detector.  The  variation  is  input  as  a  standard  deviation  based  on  experimental  data.  That 
standard  deviation  can  represent  the  inherent  variability  for  an  uncorrected  system,  or  the  residual  variations  that  remain  after 
application  of  a  correction  algorithm.  It  is  recognized  that  pattern  noise  is  frequently  a  limiting  factor  in  the  practical 
operation  of  FPA  sensors.  Due  to  its  importance,  this  area  is  receiving  further  attention  through  the  development  of  1 -point 
and  2-point  correction  algorithm  simulations  under  the  direction  of  Eustace  Dereniak  at  the  University  of  Arizona  Optical 
Sciences  Center.  Results  of  this  effort  will  be  incorporated  when  the  FPA  module  is  distributed. 


As  is  the  case  in  earlier  IT1M  sensor  models,  linear  transfer  functions  applied  in  the  frequency  domain  are  used  to 
treat  degradations  caused  by  optics,  electronics,  and  so  forth.  Two  new  effects  treated  in  the  FPA  model  are  charge  transfer 
efficiency  and  correlated  double  sampling.  The  transfer  functions  which  approximate  these  effects  are  given  below: 


MTFcte  =exP 


-m(l-  e)  1 


where 


mtfcte 

e 

m 

fHC 


MTF  for  charge  transfer  efficiency 
CCD  charge  transfer  efficiency 

number  of  charge  transfers  from  detector  to  output  amplifier 
horizontal  spatial  frequency  cutoff  of  the  array  (mrad'1) 


MTF 


CDS 


1  -  cos(2rtft) 

I  +  (2*fTD)2 
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where  MTFcdS  =  MTF  for  correlated  double  sampling 

f  =  frequency  (Hz) 

t  =  clamp- to- sample  time(s) 

Td  =  CDS  dominant  time  constant(s) 

Finally,  additional  noise  models  have  been  added  to  treat  mechanisms  found  in  CCD  structures.  The  CCD  noise 
sources  represented  are  trapping  state,  pixel  reset,  charge  transfer,  output  amplifier,  and  dark  current  noise.  For  a  well 
designed  sensor  viewing  terrestrial  scenes,  all  of  these  noise  sources  should  be  small  compared  with  the  photon  shot  noise; 
nevertheless,  they  are  treated  in  the  model  to  allow  consideration  of  low  background  scenes. 

The  FPA  model  is  currently  in  use  for  the  MET -TIM  application.  Validation  exercises  are  planned  following  the 
incorporation  of  the  new  fixed  pattern  noise  correction  model.  Distribution  of  the  model  as  a  TTIM  module  is  planned  for 
later  this  year. 

5.0  IMAGING  SEEKER  SIMULATION  EXAMPLE 

The  image  sequence  shown  as  Figure  4  illustrates  the  type  of  simulations  made  possible  using  TTIM.  In  this  case, 
TTIM  is  mated  with  the  Infrared  Modeling  and  Analysis  (IRMA)  scene  generator1^  which  produces  the  required  target  and 
background  imagery.  TTIM  introduces  the  atmospheric  effects,  models  a  PtSi  FPA  imaging  seeker  and  simulates  the 
operation  of  a  simple  gated  video  tracker.  The  six  frames  show  samples  of  the  tracker  behavior  during  the  flight.  The 
simulation  actually  considered  tracking  updates  at  the  full  30  Hz  frame  rate  of  the  postulated  sensor. 


D.H.  Goldstein,  Infrared  Modeling  and  Analysis  (TRMAi  Computer  Program  User's  Manual.  AFATL-TR-85-44, 
U.S.  Air  Force  Armament  Laboratory,  June  1985. 
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Figure  1.  TOM  3.0  vertical  structure  and  module  summary. 
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QUARTER  MOON  CLEAR  NIGHT 

Figure  3.  Sample  simulated  imagery  at  three  light  levels  for  prototype  night  goggles  viewing  a  winter  scene. 
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310.2  m 


258.0  m 


146.2  m 


Figure  4.  Closing  sequence  simulation  for  a  postulated  imaging  seeker  and  tracker.  The  tracking  gate  is  indicated 
on  each  image  and  range  to  target  is  given  below  each  image. 
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Table  1.  Image  Intensifier  Subsystem  Transfer  Function  Definitions 


1. 

Optics  -  Diffraction 

H(kx,ky)  * 

2/rt  [COS'1  (A)  -  A(1-A2H 

A 

F#  (Is2  +  Is2)’*/  f 

*0 

diffraction  wavelength  (microns) 

F. 

objective  f/# 

kA 

image  spatial  frequency  components  (mrad'1) 

f 

objective  focal  length  (mm) 

2. 

Optics  -  Blur 
H(kx,ky)  = 

e-n2w2(kx2  +  ky2) 

w  = 

e-1  half-width  of  the  optical  system  point  spread  function  (mrad) 

3. 

Optics 

User  input  table  describing  an  arbitrary  optical  system  MTF. 

4. 

Tube 

User  input  table  describing  an  arbitrary  tube  MTF. 

5. 

Eyeball  1 
H(kx,ky)  «= 

e(-2nkxyY/M)  .e(-6nkxyY/M) 

kxy 

(k„2  +  ky2)’* 

M 

system  magnification 

1 

eyeball  resolution  parameter 

6. 

Eyeball  2 
H(kx,ky)  - 

gf-Sitk^y/M) 

Table  2.  Prototype  Night  Goggles  Specification 


Parameter 

Specification 

Objective  Lens  Focal  Length 

2.6  cm 

T/Number  of  Lens 

1.5 

Eyepiece  Transmission 

0.8 

Magnification 

1.0 

Luminous  Gain  of  Image  Intensifier 

20,000 

Maximum  Brightness  of  Image  Intensifier 

2.0  fL 

Photocathode 

S-20ER 

Type  of  Image  Intensifier 

2nd  Generation 

Proximity  Focused 

Tube  Resolution 

300  Cycles/cm  Typical 

Equivalent  Brightness  Input  (EBI) 

About  2x10  ”  Im/cm2 

Filter 

None 
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ABSTRACT 

The  Multi-Mission  Area  Sensor  (MMAS)  Study  is  to  provide  improved 
target  acquisition  information  to  the  Brigade  Commander.  This  improved 
information  will  be  derived  through  the  use  of  new  types  of  sensors  and  by  the 
fusion  of  data  from  existing  brigade  sensors.  This  study  is  considering  electro- 
optical,  infrared,  radar,  acoustic  and  signal  intelligence  sources.  The  simulation 
of  the  performance  of  acoustic  sensors  is  an  important  part  of  the  MMAS 
program 

For  the  MMAS  Study,  multi-location  acoustic  sensor  systems  are  being 
simulated.  These  systems  consist  of  two  or  three  acoustic  receivers  each  of 
which  can  measure  the  intensity,  spectral  signature  and  approximate  bearing  of 
an  acoustic  target.  The  spectral  signatures  are  used  to  infer  target  type.  The 
bearing  information  obtained  from  two  or  three  receivers  is  combined  to 
triangulate  the  apparent  location  of  the  acoustic  source.  Performance  modeling 
for  these  systems  has  been  separated  :nto  four  components: 

1)  modeling  the  detectability  of  each  target  at  each  receiver  location; 

2)  modeling  the  bearing  tracking  function  for  each  target  at  each  location; 

3)  modeling  the  fusion  of  data  from  multiple  receiver  locations  to 
triangulate  the  apparent  target  positions;  and 

4)  correcting  for  the  acoustic  propagation  time  delays  to  estimate  the 
current  target  locations. 

As  part  of  the  MMAS  program,  OptiMetrics  has  developed  a  simulation  model 
which  includes  the  above  functions. 

The  paper  discusses  simulation  methodologies  used  in  the  acoustic  system 
performance  prediction  model.  Algorithms  are  presented  which  are  used  for 
smoothing  and  correlating  the  acoustic  reports  from  the  various  sensor  locations. 
Results  are  shown  which  predict  the  performance  of  the  acoustic  sensor  suite  in  a 
sample  scenario. 

The  acoustic  performance  simulation  model  described  in  the  paper  is 
applicable  to  the  Digital/Electronic  Terrain  Board  (DETB)  simulation.  Various 
model  enhancements  could  be  performed  to  improve  the  flexibility  of  the  model 
in  the  DETB  application. 


1.0  INTRODUCTION  AND  BACKGROUND 

Early  this  year  S^TO  was  designated  to  lead  the  Multi-Mission  Area  Sensor  (MMAS)  Study  to  provide  improved 
target  acquisition  information  to  the  Brigade  Commander.  This  improved  information  is  to  be  provided  by  the  use  of 
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additional  sensors  and  by  the  fusion  of  data  from  all  brigade  sensors.  Two  sensor  classes  are  to  be  addressed:  those 
presently  in  6.3  or  above,  Advanced  Development,  for  deployment  between  1996  and  2000  and  those  presently  in  6.2, 
Advanced  Research,  for  deployment  from  2000  to  2006. 

This  Study  is  to  consider  electro-optical,  infrared,  radar,  acoustic  and  signal  intelligence  sources.  A  key  component 
of  the  Study  is  the  development  and  application  of  a  sensor  performance  and  data  fusion  model  to  evaluate  performance 
trade-offs.  One  key  aspect  of  that  model  is  the  development  of  a  module  for  acoustic  sensors  and  the  fusion  of  data  from 
such  sensors  with  other  types  of  target  acquisition  information. 

For  the  simulation  part  of  the  MMAS  Study,  it  was  decided  to  use  an  enhanced  version  of  the  Intelligence  and 
Electronic  Warfare  (IEW)  Model  developed  by  TRAC-WSMR  and  its  contractors*.  That  model  was  originally  developed  to 
predict  the  performance  of  Signal  Intelligence  sensors  for  a  Corps  area  of  the  battlefield.  In  1986,  OptiMetrics  Inc.  was 
funded  by  TRAC-WSMR  to  add  the  capability  to  address  electro-optical  and  infrared  sensors  with  in  that  model^.  For  the 
MMAS  Study,  additional  sensor  models  and  fusion  models  have  had  to  be  added  to  the  IEW  Model. 

In  reality,  the  "IEW  Model"  is  not  one  model  but  a  series  of  models  which  can  be  used  in  concert  to  address  a 
particular  battlefield  scenario.  For  the  MMAS  Study  additional  sensor  modules  had  to  be  added  to  predict  the  performance  of 
air  defense  radars,  moving  target  indicator  (MTT)  ground  surveillance  radars,  and  acoustic  sensors.  In  addition,  enhancements 
were  made  to  the  EO/IR  sensor  models  to  better  model  the  time-lines  for  target  detection  and  to  improve  the  countermeasure 
modeling  capabilities.  Simulation  models  for  data  fusion  were  also  added.  A  schematic  representation  of  the  model,  as  it  is 
being  used  for  the  MMAS  Study,  is  shown  in  Figure  1.  In  the  present  paper,  we  will  discuss  only  the  acoustic  portions  of 
the  model. 

2.0  ACOUSTIC  SENSOR  SIMULATION 

For  the  MMAS  Study,  a  multi-location  acoustic  sensor  system  is  to  be  simulated.  This  multi- location  system 
consists  of  two  or  three  acoustic  receivers  each  of  which  can  measure  the  intensity,  spectral  signature  and  approximate 
bearing  of  an  acoustic  target  The  spectral  signatures  are  used  to  infer  target  type.  The  bearing  information  obtained  from 

1  K.  C.  Trott,  F.  K.  Frantz  and  D.  A.  Barnes,  "Intelligence  and  Electronic  Warfare  Movement  Simulator,"  TRASANA- 
TR-2-1986,  PAR  Government  System  Corp,  July  1985. 

^  J.  W.  Petraska,  N.  Hadwick,  T.  J.  Rogne,  "Intelligence  and  Electronic  Warfare  Model  Electro-Optical  Sensor 
Simulation  Package  -  User's  Guide,”  OMI-204,  April  1987. 
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two  or  three  receivers  is  combined  to  triangulate  the  apparent  location  of  the  acoustic  source.  The  modeling  of  the 
performance  of  this  system  was  separated  into  four  components: 

1)  modeling  the  detectability  of  each  target  at  each  receiver  location; 

2)  modeling  the  bearing  tracking  function  for  each  target  at  each  location; 

3)  modeling  the  fusion  of  data  from  multiple  receiver  locations  to  triangulate  the  apparent  target  positions;  and 

4)  correcting  for  the  acoustic  propagation  time  delays  to  estimate  the  current  target  locations. 

3.0  ACOUSTIC  DETECTION  RANGE  PREDICTIONS 

For  the  MMAS  Study,  it  was  decided  to  use  the  TACOM  Acoustic  Detection  Range  Prediction  Model  (ADRPM) 
off-line  as  a  component  of  the  acoustic  sensor  performance  predictions.  TACOM  supplied  representative  ground  vehicle 
signatures  in  the  1/3  octave  format  required  by  ADRPM.  ADRPM  could  then  be  used  in  the  standard  manner  to  perform 
range  prediction  calculations. 

ARDEC  supplied  representative  helicopter  signature  data  which  was  processed  to  the  ADRPM  format  by 
OptiMetrics.  However,  because  ADRPM  assumes  a  relatively  wide,  1/3  octave,  spectral  bands,  if  used  in  the  normal 
manner  ADRPM  would  not  correctly  predict  the  range  performance  for  the  narrow,  1  hertz,  bandwidth  helicopter  signature 
lines.  As  a  expedient  method  to  correct  for  this,  the  background  intensity  was  decreased  by  the  square  root  of  the  bandwidth 
ratio  in  those  spectral  bands.  A  companion  paper  presents  examples  of  the  target  signatures  and  the  range  predictions 
developed  for  the  study. 

The  standard  ADRPM  model  only  provides  a  measure  of  the  range  where  the  target  is  just  at  the  detection 
threshold.  For  the  present  study  it  was  required  that  some  measure  be  provided  of  the  ranges  at  which  targets  can  be 
identified  and  accurately  tracked  by  a  directional  acoustic  sensor.  Relations  have  been  developed  by  others  for  the  accuracy 
of  bearing  determination  verses  the  sensor  S/N  ratio^.  This  relationship,  called  the  Cramer-Rao  bound,  is  shown  in  Figure 
2.  Using  that  curve  we  have  computed  three  functional  ranges  for  use  in  the  model:  detection,  identification  with  6  degree 
bearing  accuracy  and  identification  with  3  degree  bearing  accuracy.  The  S/N  ratio  corresponding  to  each  of  those  ranges  is  3 
dB,  9  dB  and  13  db,  respectively.  These  functional  ranges  were  then  computed  with  ADRPM  for  each  target  and  condition. 

3  Private  Communication,  W.  Donnally,  ARDEC,  January  1989. 


When  the  MMAS  acoustic  simulation  is  run,  the  distance  is  determined  between  each  target  and  receiver  pair.  It  is 
then  determined  if  the  target  is  detectable  given  the  precomputed  range  results  from  ADRPM.  If  the  target  is  detectable,  it 
is  then  determined  if  it  is  identifiable  and  the  approximate  angular  accuracy  with  which  it  can  be  tracked.  A  Gaussian 
random  error  is  then  added  to  the  target  bearing  and  an  acoustic  detection  report  (bearing  and  identification)  is  generated  for 
later  use  with  the  fusion  model. 

4.0  ACOUSTIC  BEARING  TRACKING  ALGORITHMS 
The  initial  algorithms  for  acoustic  tracking  were  proposed  by  John  Allen  as  part  of  a  fusion  study  for  S^TO4.  The 
algorithms  were  modified  during  the  model  development  process  by  OptiMetrics'  personnel.  The  bearing  track  algorithms 
are  intended  to  correlate  and  smooth  bearing  reports  resulting  from  a  single  target  If  two  sequential  bearing  reports  fall 
within  predefined  angular  velocity  and  angular  acceleration  gates,  then  they  are  used  to  create  a  new  bearing  track  file. 
Subsequent  observations  will  be  added  to  a  track  file  when  they  fall  within  an  acceptance  bound  of  the  extrapolated  position 
of  an  existing  track.  After  three  or  more  observations  are  associated  with  a  track,  then  a  least  square  polynomial  fit  is  used 
to  smooth  and  extrapolate  the  bearing  observations  in  time. 

5.0  ACOUSTIC  POSITION  TRACKING  ALGORITHMS 
The  purpose  of  the  position  tracking  algorithms  are  to  associate  bearing  tracks  from  the  same  target  observed  from 
different  sensor  locations  to  obtain  a  triangulated  position  fix  on  each  target  The  algorithm  for  the  position  determination 
uses  the  assumptions  that  the  S/N  ratio  is  inversely  proportional  to  range  and  a  knowledge  of  the  velocity  of  sound  to 
develop  a  time  coincidence  equation  between  two  bearing  reports.  Once  that  has  been  accomplished,  the  target  position  is 
found  by  simple  triangulation.  This  simple  sounding  process  is  complicated  by  the  facts  that  all  of  the  data  includes 
random  errors  and  that  a  single  bearing  track  from  one  receiver  may  associate  with  multiple  bearing  tracks  from  another 
receiver.  The  random  errors  can  cause  missed  position  tracks  or  incorrect  associations,  while  the  multiple  associations 
cause  ghost  tracks. 

The  position  tracks  formed  are  also  smoothed  by  fitting  to  polynomials.  The  polynomials  are  used  to  establish 
gates  for  the  next  expected  locations  of  the  targets.  Because  acoustic  signals  have  finite  propagation  delays,  the  measured 


4  Private  Communication,  J.  Allen,  LICA  Systems  Inc.,  March  1989. 
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tracks  represent  past  positions  of  the  targets;  hence  the  polynomials  are  also  used  estimate  the  target  locations  at  the  present 
instant. 

6.0  SAMPLE  RESULTS  OF  ACOUSTIC  FUSION 

The  acoustic  fusion  algorithms  have  been  demonstrated  by  running  against  a  small  scenario  illustrated  in  Figure  3. 
In  that  view,  two  acoustic  receiver  locations  are  seen  at  the  left  end  of  the  terrain  grid.  These  sensors  are  attempting  to  track 
four  ground  targets  and  three  helicopters  moving  over  trajectories  as  shown.  The  ground  vehicles  are  moving  on  the  terrain 
and  the  helicopters  are  assumed  to  be  flying  at  SO  meters  above  the  local  terrain.  The  ground  vehicles  are  generally  visible 
to  the  sensors;  however  line-of-sight  to  the  helicopters  is  often  restricted  by  the  terrain. 

The  apparent  positions  determine  by  the  acoustic  position  tracking  algorithms  is  shown  in  Figures  4  and  S.  It  can 
be  seen  from  this  plan  view,  that  the  model  shows  that  both  the  ground  vehicles  and  the  helicopters  are  tracked  over  most  of 
their  trajectories  by  the  acoustic  sensors.  Where  the  targets  are  not  tracked,  it  is  mostly  due  to  range  and  terrain  obscuration 
which  lowers  the  target  signal  levels  below  the  background  noise  level.  It  is  also  apparent  from  the  figures  that  there  are  a 
number  of  position  tracks  where  there  are  no  targets.  These  are  ghost  tracks  resulting  from  incorrect  associations  between 
bearing  tracks  from  the  two  acoustic  receivers.  It  is  believed  that  many  of  those  tracks  can  be  eliminated  through 
correlations  with  true  tracks  and  through  the  use  of  realistic  velocity  gates;  however,  those  algorithms  have  not  yet  been 
implemented  in  the  model. 

7.0  ADDITIONAL  FUSION  LEVELS 

The  fusion  of  the  multi-location  acoustic  data  is  only  the  First  level  of  fusion  envisioned  in  the  MMAS  Study.  In 
addition,  fusion  of  multiple  types  of  sensor  data  is  to  be  accomplished.  Figure  6.  shows  the  fusion  structure  being 
implemented  in  the  MMAS  program.  The  overall  structure  was  developed  by  a  contractor  team  headed  by  LICA  Systems 
Inc.  for  S^TO  5.  That  structure  represents  a  three  level  approach,  with  the  Fust  level  being  the  fusion  of  bearing  only 
sensors  to  get  position  data.  The  bearing  only  sensors  include  the  acoustic  sensors  and  signal  intelligence  sensors.  The 
next  level  of  fusion  will  correlate  reports  from  the  sensors  assigned  within  a  single  battleField  functional  area.  The 
battlefield  functional  areas  (BFA's)  addressed  include:  air  defense,  fire  support,  maneuver  control  and  IEW.  The  air  defense 

^  J.  Allen,  G.  Wittenberg,  G.  Fox,  et.  al.,  "Functional  Architecture  Description  of  the  Force  Level  Integration  Segment 
for  the  Multi-Mission  Area  Sensor  Study,"  LICA  Systems,  Inc.,  SAIC,  TRW  Federal  Systems  Group,  Draft  Report 
prepared  for  S^TO,  Harry  Diamond  Laboratories,  January  1989. 
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node  represents  the  most  challenging  for  fusion,  because  the  time  lines  are  short  and  the  targets  may  be  observed  with 
multiple  sensors.  It  is  proposed  that  a  multi-hypothesis  tracking  algorithm  will  be  used  for  BFA  fusion  at  the  air  defense 
node. 

For  the  other  BFA  nodes  and  at  the  Brigade  level  (Functional  Level  Integration)  spatial  and  temporal  correlation 
will  be  used  for  data  fusion.  Basically,  target  reports  at  those  levels  will  be  associated  when  they  meet  correlation  gates 
which  imply  they  are  the  same  target,  even  though  they  have  been  reported  by  various  types  of  sensors.  Track  files  will  be 
obtained  and  updated  for  such  reports  in  a  manner  similar  to  the  method  described  for  the  position  reports  from  the  acoustic 
sensors. 

8.0  MEASURES  OF  EFFECTIVENESS 

A  new  module  has  been  developed  to  calculate  measures  of  effectiveness  for  the  MMAS  Study.  The  measures  of 
effectiveness  are  calculated  for  one-on-one,  one-on-many  and  many-on-many  sensor  and  target  combinations.  The  measures 
of  effectiveness  calculated  include: 

maxim  -an  detection,  recognition  and  identification  ranges  for  each  type  of  target; 

accuracy  of  location; 

range  at  which  location  accuracy  of  a  defined  level  is  obtained; 

percentage  of  time  a  target  is  detected  within  a  critical  area; 

time  to  report  a  target;  and 

time  that  target  report  leads  or  lags  target's  entry  into  a  critical  area. 

Various  options  are  also  available  for  averaging  those  statistics  over  classes  of  targets  and  sensors.  One  difficulty 
with  the  MMAS  Study  is  how  to  measure  the  effectiveness  of  the  fusion  algorithms.  The  fusion  algorithms  do  not 
generally  lead  to  the  detection  of  more  targets,  but  rather  improve  the  quality  of  information  about  those  targets  which  are 
detected  and  minimize  confusion  generated  by  multiple  detections  of  the  same  target.  Hence  it  is  anticipated  that  additional 
measures  of  effectiveness  will  be  developed  to  quantify  those  factors. 

9.0  STATUS  AND  FUTURE  PLANS 

The  MMAS  sensor  models  and  the  fusion  model  are  operational  on  OptiMetrics'  Micro- VAX  II  computer.  The 
models  will  soon  be  adapted  to  the  Silicon  Graphics  4/D  computer  workstation.  Predictions  have  been  completed  for  the 
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MMAS  Demonstration  Experiment  which  is  presently  planned  for  June  of  1990.  The  model  is  presently  being  used  to 
predict  MMAS  system  performance  in  a  Brigade  area  scenario  set  in  Europe  in  1995.  Plans  are  underway  to  add  cross¬ 
sensor  cuing  to  the  model. 
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Figure  5.  Modeled  helicopter  tracking  from  sample  scenario  (realistic  errors  assumed  for  bearing  accuracy  > 


Figure  6.  Multi-level  fusion  architecture  for  the  MMAS  program. 
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AUTOMATIC  TARGET  RECOGNITION  OF  SUB  PIXELLAR  TARGETS 


Conrad  H.  Hoeppner 
AAI  Corporation 

The  detection,  recognition  and  classification  of  objects  which  are  potential 
targets  by  means  of  non- imaging  techniques,  which  are  completely  passive,  represents  a 
new  field  with  a  tremendous  potential.  AAI  Corporation  has  pioneered  in  this  field 
classifying  objects  based  on  their  modulated  exitance.  Exitance  is  the  term  used  to 
denote  the  radiation  of  energy  from  an  object  by  virtue  of  its  temperature  being  above 
absolute  zero.  Its  units  are  watts  per  square  centimeter  per  micrometer  of  bandwidth. 
This  is  the  familiar  blackbody  radiation  but  reduced  by  the  surface  emissivity.  The 
object  does  not  "cool  off"  as  a  result  of  this  energy  emission  because  surrounding 
objects  radiate  energy  to  it.  Nonetheless,  the  object  is  radiating  energy  which  can  be 
detected  at  great  distances.  If  this  exitance  is  changed  in  some  manner,  either 
statically  or  dynamically,  by  means  peculiar  to  the  radiating  object,  such  that  it  is 
different  than  the  exitance  from  the  surrounding  terrain,  the  object  can  be  detected, 
recognized  and  classified.  An  example,  in  the  visible  spectrum  is  the  digital  color 
temperature  thermometer.  Another  example,  with  reflected  radiation  again  in  the  visible 
spectrum,  an  object  may  be  recognized  if  painted  red  while  the  background  is  green.  This 
same  principle  holds  true  for  the  cooler  objects  radiating  in  the  far  infrared.  They 
may  be  recognized  by  the  selective  emissions  of  their  exitance. 

AAI  has  constructed  equipment  and  conducted  tests  in  the  far  infrared  region 
(7-14  microns)  detecting  and  recognizing  road  vehicles  of  various  kinds.  Low  cost 
pyroelectric  room  temperature  sensors  were  used  in  conjunction  with  first  surface 
Cassegrain  optics  for  wide  band  energy  reception.  AAI's  location  is  extremely  favorable 
for  these  tests,  being  next  to  a  rock  quarry  and  crushing  operation  where  various 
vehicles  are  moving  in  a  confined  space,  foliage  is  in  place  for  penetration  tests  and 
a  foreground  consisting  of  an  AAI  parking  lot  makes  it  possible  to  run  tests  on  vehicles 
in  AAI  inventory  such  as  tanks  and  armored  personnel  carriers  without  leaving  the 
protected  area  of  AAI  property.  Consequently,  the  tests  were  made  in  this  area;  some 
tests  through  foliage,  some  with  unrestricted  vision  and  some  in  fog  and  rock  dust  in 
which  the  objects  could  not  be  discerned  visibly.  The  wide  band  optics  of  the  first 
surface  Cassegrain  telescope  was  employed  for  these  tests.  The  system  may  be  described 
as  follows. 


The  system  utilizes  a  unique  technique  for  locating  and  identifying  armored 
vehicles,  including  tanks,  in  the  modem  battlefield.  Using  analysis  techniques  adapted 
from  anti-submarine  warfare,  the  approach  differentiates  a  tank  from  its  surroundings 
by  detecting  its  emission  and  reflections  and  the  modulation  of  those  emissions  and 
reflections  caused  by  movement  (vibration)  of  the  tank  parts  and  surfaces.  These 
vibrational  target  characteristics  are  used  to  locate  and  classify  the  target  with  great 
accuracy . 
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The  basic  approach  is  to  examine  the  modulation  spectrum  of  the  infrared 
(IR)  energy  from  the  target  vehicle  by  exploring  the  vibrations  of  the  tank  exterior 
and/or  the  motion  of  the  exterior  parts  with  either  active  or  passive  vibration/motion 
detectors.  While  these  two  methods  are  complementary,  it  is  expected  that  the  active 
method  would  not  be  used  when  it  is  desired  to  remain  covert.  Both  methods  (active  and 
passive)  utilize  analysis  techniques  employed  in  anti-submarine  warfare  detection  and 
identification,  rather  than  optical  type  pattern  recognition. 

The  same  system  is  applicable  to  the  detection  and  classification  of 
aircraft  and  ships. 

Contemporary  Automatic  Target  Recognition  (ATR)  is  largely  directed  toward 
geometric  pattern  recognition  using  models,  matched  filters,  neural  networks,  etc.  All 
are  spatially  oriented  and  all  require  multiple  pixels  (50  to  100)  to  inspect  the 
target.  The  range  of  operation  is,  therefore,  limited  by  a  combination  of  pixel  size 
and  target  size.  It  is  also  necessary  that  there  be  thermal  contrast  between  target 
and  background  to  determine  the  geometric  pattern  of  the  target. 

Other  methods  of  detecting,  recognizing,  identifying  and  classifying  targets 
are  available  and  should  be  considered  in  the  ATR  system.  Techniques  are  presented  here 
which  extend  the  effectiveness  of  ATR  and  at  the  same  time  simplify  the  process.  These 
techniques  are  temporal  rather  than  spatial  and  use  ATR  techniques  to  get  a  "signature" 
much  as  is  done  in  sonar,  geophysical  prospecting  and  speech  recognition.  The  movement 
of  the  target  into,  within  and  emerging  from  a  pixel  is  characterized  by  both  the  shape 
of  the  target  and  its  motion.  Motion  may  include  linear  translation,  rotation,  linear 
vibration  or  angular  vibration.  With  narrow  bandwidth  filters  and  parallel  processing, 
the  signal  to  noise  ratio  may  be  greatly  improved  over  that  obtained  by  scanning  with 
starring  arrays.  This  makes  possible  ATR  with  targets  occupying  only  a  small  part  of 
a  pixel.  When  the  target  is  not  in  motion,  equivalent  relative  movement  may  be  produced 
by  a  mini  scan  of  the  optics  (less  than  one  pixel  up  to  two  pixels  in  amplitude) , 
producing  effect  similar  to  target  motion.  Under  many  circumstances,  distance  to  target 
and  target  velocity  may  also  be  obtained.  Also,  under  many  conditions,  thermal  contract 
between  target  and  background  is  not  required. 

The  task  of  gathering  background  data  for  geometrical  target  pattern 
recognition,  now  just  beginning,  Is  a  monumental  one.  Both  targets  and  background 
(clutter)  must  be  catalogued.  For  temporal  signatures,  the  complexity  is  reduced  by 
several  orders  of  magnitude.  Most  background  clutter  does  not  move  or  vibrate. 

The  ability  of  the  sub-pixellar  system  to  penetrate  foliage  is  phenomenal. 
Shown  in  Figure  1  is  a  photograph  of  the  area  being  inspected.  Figure  2  and  Figure  3 
show  signal  analyzer  records  of  automobiles  and  trucks  which  passed  on  the  far  side  of 
the  foliage.  Figures  8  through  31  are  detector  output  voltages  obtained  from  various 
targets  at  ranges  to  2.8  Km. 

Dynamic  Targets  in  the  Digital/Electronlc  Terrain  Board 

Targets  with  characteristic  translation,  rotation  and  reciprocal 
translation- rotation  should  be  included  in  the  terrain  board.  Developing  technology 
portends  far  more  economical  and  definitive  target  recognition  and  classification 
methods  than  does  the  presently  pursued  geometric  pattern  recognition.  These  new 
methods,  however,  require  that  the  target  be  moving  or  vibrating  in  many  cases. 
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BRUEL  &  KJAER  HIGH  RESOLUTION  ANALYZER  TYPE  2033 
EXPONENTIAL  AVERAGING  DYNAMIC  RANGE:  80dB 

FIGURE  2 
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BRUEL  &  KJAER  HIGH  RESOLUTION  ANALYZER  TYPE  2033 
EXPONENTIAL  AVERAGING  DYNAMIC  RANGE:  80dB 

FIGURE  3 


However,  they  should  not  be  limited  to  "stand  alone"  use  but  may  be  combined  with 
imaging  systems.  The  methods  of  performing  automatic  target  recognition  by  means  of 
vibration  analysis  are  often  simple  and  direct.  They  can  be  employed  with  either 
optical  or  radio  frequency  systems  and  may  operate  in  an  active,  semi-active  or  passive 
manner.  The  system  described  herein  is  a  passive  optical  system  utilizing  first  surface 
Cassegrain  optics,  an  8  -  12  micrometer  band  pass  filter  and  two  pyroelectric  detectors. 
The  data,  displayed  in  graphic  form  as  shown  here,  has  been  taken  using  this  system. 

Another  important  consideration  is  that  typical  targets  will  be  camouflaged 
and  hidden,  making  geometric  pattern  recognition  quite  useless. 

Vibration  Signatures  May  be  Obtained  from  a  Small  Spot  of  the  Target 

Refer  again  to  Figure  1.  In  the  two  foot  square  opening  in  the  leaves,  a 
portion  of  the  rear  end  of  an  automobile  will  be  noted.  It  was  through  this  aperture 
that  much  of  the  data  was  taken.  This  is  another  feature  to  be  included  in  the  terrain 
board. 


IFF  is  still  another  feature  to  be  considered.  Methods  of  modulating 
exitance  with  the  "code  of  the  day"  are  simple  and  inexpensive. 

One  example  of  the  IFF  unit  is  an  electronic  shutter  which  alternately 
transmits  and  diffuses  the  exitance  from  a  portion  of  the  vehicle.  The  chopping  code 
may  be  vary  extensive  and  precise  to  allow  correlation  deep  into  noise,  another  example 
is  the  use  of  two  shutters,  each  used  in  conjunction  with  a  polarizer.  At  one  shutter, 
the  exitance  would  be  polarized  vertically  and  at  the  other  horizontally.  Switching 
them  alternately  maintains  the  total  exitance  at  a  constant  level  but  permits  a  dual 
polarized  detector  to  differentiate  and  receive  the  identification  code.  These  units 
may  be  controlled  electrically  to  be  silent  or  active  at  the  discretion  of  the  vehicle 
operator. 


Target  detection  and  recognition  may  be  considered  at  three  levels: 

1.  The  target  is  larger  than  the  pixels  and  many  pixels  may  scan  the 
target.  This  is  the  case  used  for  geometric  image  analysis. 

2.  The  target  and  pixel  are  approximately  the  same  size. 

3.  The  target  is  smaller  than  each  pixel  and  may  be  lost  between  pixels. 


The  first  case  is  well  known.  Target  motion  and  vibration  signatures  may 
be  obtained  in  much  the  same  way  as  in  geometric  image  analysis  and  obtained  in  addition 
to  it. 

The  second  case  represents  an  extension  of  the  capabilities  of  the  system 
to  classify  targets  which  previously  could  not  be  classified  by  the  image  analysis 
system.  An  example  of  how  the  technique  is  used  is  shown  in  Figures  4  and  5. 

In  the  third  case,  when  the  target  may  be  lost  between  pixels  because  of 
its  small  size,  it  may  still  be  detected  and  classified  if  it  moves  through  one  or  more 
pixels.  Only  if  it  is  stationary  or  moving  radially  with  respect  to  the  detector  will 
it  not  be  detected.  For  this  situation,  the  pixel  pattern  may  be  translated  or  rotated. 
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The  multi -microbeam  telescope  described  later  can  be  converted  very  simply  into  a 
mini-  scan  system  by  moving  either  the  secondary  or  tertiary  mirrors.  They  may  be  moved 
in  a  vibratory  manner  or  in  discrete  steps,  but  in  either  event,  the  entire  volume  of 
the  beam  of  the  telescope  is  inspected.  It  is  only  necessary  to  have  the  target 
penetrate  a  single  pixel  to  obtain  a  temporal  signature. 

The  effects  which  produce  exitance  signatures  are  numerous  and  varied.  The 
salient  requirement  is  that  the  signature  be  above  the  noise  level  of  the  detector. 
This  noise  level  may  be  considered  in  two  parts:  that  inherent  in  the  detector  (Johnson 
noise)  and  that  produced  by  the  atmosphere  modulating  the  exitance.  In  the  latter  noise 
contribution,  the  atmosphere  may  modulate  exitance  from  the  background  or  change  the 
modulated  exitance  from  the  target.  They  are  expected  to  be  of  low  frequency  (0-5  Hz) 
and  will  be  removed  by  the  narrow  band  filters.  Some  of  the  effects  which  produce 
signatures  are  listed  in  Table  1.  It  should  be  noted  that  most  do  not  require  that 
there  be  contrast  between  target  and  background- -a  requirement  in  geometrical  imaging. 

When  it  is  desired  to  find  a  target  in  clutter,  any  or  all  of  the  modes  may 
be  exploited.  It  is  not  necessary  that  the  target  have  a  net  translation  with  respect 
to  the  background  to  be  detected.  It  my  be  a  reciprocal  linear  motion,  a  reciprocal 
rotational  motion  or  a  continuous  rotational  motion.  In  this  respect,  the  technique 
described  is  different  from  the  moving  target  indicating  (MTI)  radar  which  requires  that 
the  target  change  its  location  with  respect  to  the  background.  Net  translational  motion 
may  also  be  detected  by  the  system  and  differentiated  from  the  reciprocal  motion  or 
vibration.  The  multi -microbeam  and  mini-scan,  multi -microbeam  telescopes  perform  this 
function. 


It  should  be  noted  that  the  two  functions  requiring  contrast  with  the 
background  may  have  limitations  in  some  circumstances,  but  in  other  circumstances,  these 
may  be  overcome.  For  example,  if  it  is  desired  to  measure  the  vertical  vibrations  of 
a  nearby  vehicle  against  a  distant  background,  then  a  band  pass  filter  is  chosen  for 
the  optical  system  at  a  wavelength  at  which  there  is  atmospheric  attenuation.  The 
contrast  is  then  greatly  enhanced  by  the  atmospheric  absorption  difference  between  the 
short  path  and  the  long  path.  The  detector  is  sensitive  at  all  wavelengths  and  the  band 
pass  filter  imposes  limits  on  its  reception  of  energy.  If  the  vibration  measurement 
is  being  made  with  the  sky  as  a  background,  then  far  infrared  is  chosen  in  which  there 
is  essentially  no  sky  radiation. 

In  the  category  for  which  no  optical  contrast  with  the  background  is 
required,  a  very  powerful  means  of  target  detection  and  identification  is  obtained. 
The  great  linear  dynamic  range  of  the  sensors  (LiTa03  is  1012  to  1)  permits  nearly  any 
degree  of  steady  energy  to  fall  upon  the  sensor  without  producing  an  output  and  without 
affecting  its  sensitivity  to  changing  (or  vibrating)  energy.  The  low  frequency  response 
can  be  made  to  cut  off  at  any  chosen  frequency.  Also,  the  sensors  are  used  in  pairs 
connected  with  opposing  polarity  such  that  any  general  or  unfocused  energy  which  falls 
on  both  sensors  cancels.  Thus,  a  large  area  may  be  inspected  and  only  if  there  is  a 
vibrating  object  present  will  there  be  output  from  the  detecting  system.  Multiple 
vibrating  objects  may  be  detected  in  a  single  area  and  then  differentiated  by  analysis 
of  their  vibration  "signatures". 

Targets  may  be  identified  by  the  shape  of  the  detector  output  as  the  target 
enters  and  leaves  a  pixel.  This  is  shown  in  Figures  4  and  5. 


Moving  into  and  out  of  beam 
Moving  forward  and  backward 


tckgrotmd  Contrast  With  Target  Required 
Doppler  shift  caused  by  vibration 
Polarization  caused  by  vibration 

(a)  Change  of  surface  angle 

(b)  Stretching  and  compressing 

Heating  and  cooling  caused  by  stretching  and  compressing 
Emissivity  change  caused  by  stretching  and  compressing 
Interferometer 

Target  tilting  or  turning  to  expose  new  surfaces  at  different 
temperatures 


TABLE  1 


THE  MODULATION  OF  EXITANCE 


ed  area,  whl^h  Is  related  to  shape,  of  a  target  moving  into  a  pixel 
temporal  signature  as  shown  In  Figure  4. 
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Temporal  Signature  of  a  Vibrating  and  Translating  Target 


TECHNICAL  DISCUSSION 


Characterization  by  Modulated  Exitance 

Targets  such  as  tanks,  transports  and  aircraft  will  be  found  to  possess 
characteristic  structural  and  vibrational  modes  containing  distinctive  temporal 
frequencies  which  affect  the  emitted  infrared  radiation  field;  this  is  the  case  just 
as  ships  and  submarines  emit  distinctive  sonar  frequencies  and  tactical  land  vehicles 
and  air  vehicles  emit  unique  combinations  of  sound  frequencies.  Infrared  signatures 
of  vehicles  of  interest  can  be  obtained  either  by  direct  measurement  (empirically)  or 
by  analysis.  Such  signatures  could  be  developed  for  all  targets  both  with  and  without 
specific  payloads. 

Exitance  is  merely  the  self  emittance  of  an  IR  source  arising  from  its 
absolute  temperature.  Modulated  exitance,  in  the  context  of  the  AAI  concept,  refers 
to  the  changes  in  the  radiated  pattern  due  to  structural,  motional  and  thermal  changes 
in  the  object  being  considered. 

The  basic  approach  for  modulated  exitance  detection  is  to  examine  the 
modulation  spectrum  of  the  emitted  infrared  (IR)  energy.  Amplitude  modulation  of  an 
IR  signal  from  a  tactical  target  is  produced  by  moving  parts  when  they  present  different 
surfaces  or  surface  temperature  changes  to  the  direction  of  the  sensor.  Other  amplitude 
modulation  sources  may  be  exhaust  gases,  reflected  radiation  from  other  sources  (such 
as  the  sun),  changing  of  aspect  angles  of  any  parts  and  so  forth.  These  are  relatively 
large  variations  that  are  readily  detectable  and  which  may  be  used  to  characterize  and 
identify  the  source. 

If  the  contrast  between  the  target  and  the  background  is  significant,  motion 
of  the  surface  of  the  target  with  respect  to  the  field-of -view  (FOV)  of  an  individual 
detector  is  sufficient  to  modulate  the  detected  signal.  This  motion  may  be  either 
vibration  induced  variation  of  the  surface  area  viewed  by  a  detector  or  by  similarly 
induced  motion  forward  and  backward  between  the  emitting  surface  and  the  aperture  of 
the  detector's  optical  system. 

Little  or  no  background  contrast  is  needed  to  detect  certain  kinds  of 
exitance  modulation.  These  include: 

Doppler  shift  of  the  emitted  radiation  caused  by  vibration  induced 
relative  motion 

.  Polarization  of  the  emitted  radiation  caused  by  changes  in  the  angle 
of  the  surface  or  by  changes  in  the  state  of  stress  on  the  surface 

Heating  and  cooling  caused  by  straining  and  relieving  the  surface 

Changes  in  the  emissivity  of  the  surface  caused  by  stretching  and 
compressing 

Interferometry  effects 
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.  Target  facet  rotation  exposing  surfaces  at  different  temperatures . 

Each  of  these  mechanisms  can  be  related  to  vibration  and  causes  the  received 
steady  state  signal  to  be  altered  by  the  addition  of  some  transient  component.  These 
transient  signal  components  consist  of  the  spectral  attributes  (frequencies)  that 
characterize  the  source. 

Characterization  Using  an  Active  Optical  System 

Enhanced  detection  of  certain  kinds  of  vehicle  modal  and  vibration  response 
is  achievable  using  an  active  system  based  on  solid  state  laser  technology.  Gallium- 
aluminum-  arsenide  (GaAlAs)  lasers  may  be  modulated  into  the  gigahertz  region.  Detectors 
with  matching  response  capabilities  are  similarly  available.  Thus,  it  is  possible  to 
probe  specific  small  areas  within  a  large  target  using  a  collimated  beam  to  determine 
the  frequencies  and  displacements  of  selected  structural  elements  to  extreme  precision. 
The  wavelengths  involved  would  be  outside  the  visible  band,  around  850  run,  so  some 
degree  of  covertness  is  available. 

Other  types  of  processes  related  to  optical  schemes  are  also  potentially 
useful:  thermographic  and  photographic  techniques. 

Modulated  Exitance  Detection  System 

AAI  is  developing  a  novel  passive  IR  detection  concept  which  is  adaptable 
to  a  variety  of  missions  including  detection  and  tracking  of  tactical  targets  and 
evaluation  and  characterization  of  structures  (including  vehicles) . 

The  basic  approach  is  to  examine  the  modulation  spectrum  of  the  infrared 
(IR)  energy  from  the  target  vehicle  by  examining  the  vibrations  of  the  vehicle's 
exterior  parts  with  passive  vibration/motion  detectors.  This  passive  method  of 
detecting  the  modulations  in  IR  signatures  has  the  advantages  of  being  covert  and 
simple;  in  addition,  it  has  promise  of  being  low  cost,  small,  and  lightweight. 

The  radiated  power  from  a  blackbody  source  is  not  small;  a  10  square  meter 
surface  at  ambient  temperature  emits  about  2,000  watts.  Thus,  even  though  the  amplitude 
of  the  modulated  thermal  emission  may  represent  a  small  fraction  of  the  total  signal, 
it  may  nevertheless  still  be  sensed  by  an  IR  detector  system. 

Two  factors  produce  modulation  of  the  received  radiation:  the  vibration  of 
the  object  and  the  motion  of  the  object  through  the  detection  beam  pattern  of  a 
multi -microbeam  receiving  antenna.  The  antenna  does  not  scan  but  is  held  stationary 
against  the  background  in  the  field  of  view.  The  vibrations  of  the  object  are  detected 
by  motion  in  a  microbeam  using  frequency  selective  filters;  while  its  cross  range  speed, 
for  example,  is  determined  by  the  rate  at  which  it  crosses  the  microbeams.  There  are 
several  mechanisms  that  produce  the  modulation  detectable  by  a  microbeam  sensor  pattern. 
Each  of  these  effects  is  potentially  detectable  using  the  passive  multibeam  method 
conceived  by  AAI.  A  key  aspect  that  greatly  enhances  the  detection  of  these  emissions 
is  that  they  are  transient  or  temporally  variable  so  that  sensitive  frequency  based 
detection  methods  may  be  used. 
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AAI  Prototype  System 


The  detection  of  a  vehicle  using  this  technique  may  be  accomplished  with 
a  Fast  Fourier  Spectrum  analysis  of  the  sensor  output  signal  for  frequencies  from  nearly 
dc  to  100  Hz.  Any  frequencies  within  the  band  constitutes  a  detection  signal.  Certain 
specific  bands  with  ratios  of  the  signals  received  constitutes  recognition.  A  fine 
structure  analysis  of  all  frequencies  constitutes  identification.  AAI  has  already 
developed  and  constructed  such  a  recognition  system.  The  frequencies  of  interest  are 
shown  on  Table  2. 

Using  0.1  seconds  for  analysis,  the  frequencies  from  25  to  200  Hz  will  register 
on  the  signal  analyzer  and  a  detection  occurs.  To  establish  that  the  detected  target 
is  a  vehicle  (recognition) ,  certain  discrete  frequencies  are  selected  and  the  ratios 
of  the  signal  levels  within  these  subbands  are  analyzed.  This  process  may  take  0.5  to 
1  seconds.  The  process  of  identifying  the  target  as  a  particular  vehicle  may  require 
an  additional  5  seconds. 

Measurements  made  at  AAI  show  the  feasibility  of  sensing  far  IR  energy 
emitted  from  an  object  to  determine  the  surface  vibrations  of  that  object.  These  tests 
used  a  passive  IR  sensor  to  measure  the  vibration  of  a  2  inch  diameter  thermal  energy 
source.  Vibrations  from  8  to  30  Hz  were  detected  and  observed  on  an  oscilloscope  with 
a  signal -to -noise  ratio  (SNR)  of  20  to  30  dB.  The  thermal  energy  source  used  was  the 
center  dome  of  a  conventional  loud  speaker,  heated  by  a  remote  projector  lamp.  The 
passive  sensor  was  a  lithium  tantalate  pyroelectric  sensor  with  a  7  to  14  bandpass 
filter.  The  sensor  was  used  with  a  7 -inch  diameter  reflective  telescope.  The  distance 
between  the  sensor  and  the  heat  source  was  16  feet. 

The  speaker  was  driven  by  a  signal  generator  at  an  amplitude  of  0  to  0.1 
inch  peak-to-peak  and  a  frequency  of  8  to  30  Hz.  No  signal  was  received  when  the 
speaker  vibrated  while  at  ambient  temperature.  The  projector  lamp  was  used  to  heat  the 
center  dome  of  the  speaker.  Over  a  period  of  45  seconds,  the  output  of  the  sensor 
increased  from  zero  to  full  scale,  and  then  decreased  again  to  zero  over  the  same  period 
when  the  lamp  was  turned  off.  The  maximum  temperature  change  was  approximately  20 
degrees,  from  75°F  to  95°F.  Several  additional  tests  confirmed  these  results. 

At  a  constant  frequency  of  10  Hz,  the  amplitude  of  vibration  was  changed 
from  0  to  0.1  inches  peak-to-peak.  The  signal  was  discernable  at  an  amplitude  of  0.0001 
inch  peak-to-peak.  The  bandwidth  of  the  amplifier  was  0  to  30  Hz. 

At  a  constant  amplitude  and  constant  frequency,  the  modulation  was  changed 
from  sinusoidal  to  square.  The  output  signal  changed  from  sinusoidal  to  triangular, 
reflecting  the  30  Hz  bandwidth  of  the  amplifier. 

At  a  constant  amplitude,  the  frequency  was  changed  from  8  to  30  Hz.  A  sine 
wave  was  preserved  throughout,  and  the  amplitude  of  the  signal  from  the  sensor  decreased 
to  one-half,  also  reflecting  the  30  Hz  bandwidth  of  the  amplifier. 

Figure  6  shows  the  micro-beam  receiving  telescope  used  in  the  tests.  The 
oscilloscope  pattern  shown  in  this  figure  was  stable  before  the  photographer's  flash. 
The  voltage  SNR  was  100  to  1  (40  dB)  with  the  wide  band  (0.5  to  50  Hz)  receiver.  The 
pyroelectric  sensor  is  visible  on  the  circuit  board.  It  has  a  band  pass  filter  that 
passes  frequencies  from  2.1  x  1013  to  4.3  X  1013  Hz  (7  to  14  p  wavelength). 
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Selected  Frequency  Bands  (Hertz) 


2-3 

9-11 

24-27 

53-58 

91-  98 

147-157 

3-4 

11-13 

27-30 

58-63 

98-105 

157-167 

4-5 

13-15 

30-33 

63-68 

105-113 

167-178 

5-6 

15-17 

33-37 

68-73 

113-121 

178-187 

6-7 

17-19 

41-45 

73-79 

121-129 

187-197 

7-8 

19-21 

45-49 

79-85 

129-138 

197-207 

8-9 

21-24 

49-53 

85-91 

138-147 

TABLE  2 

FREQUENCY  BANDVIDTHS  FOR  VIBRATION  ANALYSIS 
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FIGURE  6  MICROflEAM  TELESCOPE 


Figure  7  shows  the  vibrating  heat  source  used  with  the  micro-beam  receiver. 
In  the  photograph  the  projector  lamp  is  on,  but  when  the  receiving  photograph  was  taken, 
the  projector  lamp  was  off.  The  speaker  cone  was  heated  approximately  5°F  above 
ambient.  The  temperature  of  the  spot  returned  to  ambient  in  about  forty-five  seconds. 

System  Optical  Implementation 

Detection  of  modulated  IR  emissions  is  enhanced  if  the  periodic  changes  in 
the  radiated  field  couples  the  flux  collected  by  the  optical  aperture  to  a  pair  of 
differentially  connected  detectors .  Thus ,  one  detector  is  illuminated  during  one  half 
cycle  of  a  periodic  displacement  and  the  other  detector  is  illuminated  during  the  other 
half  cycle.  In  this  arrangement,  the  signal  appearing  at  the  output  of  the  detector- 
pair  circuit  contains  the  information  about  the  motion  of  the  emitting  surface  while 
any  steady  state,  common  mode  component  is  easily  rejected. 

Several  practical  means  are  available  to  produce  this  arrangement.  One 
method  under  study  at  AAI  involves  the  multi -microbeam  telescope.  In  this  approach, 
a  masking  structure  consisting  of  an  array  of  apertures  is  located  at  the  telescope 
focal  plane.  Radiation  passing  through  this  mask  is  later  refocused  onto  a  pair  of 
detectors.  When  viewed  from  the  detectors,  the  telescope  FOV  is  subdivided  into  an 
array  of  small  fields  of  view;  a  pair  of  detectors  at  the  focal  plane  would  each  see 
a  slightly  different  collection  of  pieces  of  the  overall  FOV.  For  example,  the  mask 
is  constructed  so  that  each  detector  sees  radiation  from  alternating  segments  within 
the  telescope  FOV,  like  red  or  black  squares  on  a  checker  board. 

An  extended  source  viewed  (at  relatively  close  range)  with  such  a  system 
would  have  its  image  subdivided  so  that  the  detectors  would  become  very  sensitive  to 
local  surface  exitance  effects  produced  by  structural  or  mechanical  vibration.  These 
include  changes  in  temperature,  polarization  or  emissivity,  or  other  changes  such  as 
aspect  angle  or  similar  effects.  These  effects  could  facilitate  detection  since  the 
differentially  connected  detectors  would  produce  a  signal  containing  the  same 
frequencies  as  the  mechanical  perturbations  within  the  emitting  structure. 

Optical  Detectors 

Detection  of  IR  radiation  has  been  the  subject  of  intense  scientific 
development  during  the  last  10  to  20  years.  A  variety  of  detector  materials  and  types 
have  evolved;  typically  each  type  is  targeted  at  one  of  the  atmospheric  IR  windows  or 
at  some  related  technology  (the  wavelengths  of  significant  or  important  lasers,  for 
example) . 

Pyroelectric  detectors  have  undergone  development  as  part  of  this  general 
interest  in  IR  technology.  These  detectors  are  of  potential  interest  in  the  present 
case  because  they  do  not  require  cooling;  and,  in  addition,  they  have  essentially  flat 
response  throughout  the  IR  band.  Elimination  of  the  cooling  requirement  is  of  special 
interest  because  of  the  resulting  simplification  and  reduced  size  and  cost. 
Pyroelectric  detectors  come  with  drawbacks  compared  to  many  other  kinds  of  IR  detectors. 
Except  for  very  limited  optical  modulation  or  chopping  temporal  frequencies,  the 
detectivity  (D*)  of  pyroelectrics,  the  principal  figure  of  merit  for  IR  detectors, 
cannot  compare  with  that  of  cooled  detectors. 
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FIGURE  7  VIBRATING  HEAT  SOURCE 


Pyroelectric  detectors  are  competitive  from  near  dc  to  around  1  kHz 
with  optimum  capability  usually  around  100  Hz.  Fortunately  this  is  precisely 
the  frequency  band  of  interest.  For  this  reason,  along  with  the  inherent 
importance  of  not  requiring  cooling,  pyroelectrics  are  potentially  of  major 
importance . 

While  being  more  costly,  cooled  detectors  offer  performance  benefits 
that  must  be  considered.  There  are  several  good  choices  for  the  present 
application:  photoconductive  HgCdTe  operated  at  both  LN2  (77K)  or  TE  (195K) , 
photovoltaic  HgCdTe  operated  at  77K,  InSb  (usually  operated  at  77K)  ,  and  the  lead 
salts  (PbS  and  PbSe)  that  are  available  for  use  from  ambient  down  to  77K.  Each 
of  these  has  a  better  D*  than  any  pyroelectric  device,  and  a  much  greater  optical 
modulation  bandwidth.  Thermoelectric  (TE)  coolers  require  the  least  impact  on 
the  overall  design  and  packaging;  liquid  nitrogen  (LN2)  cooling,  while  achievable 
in  closed  cycle  processes,  could  ultimately  be  difficult  to  accomplish  covertly. 

Another  cooled  detector  technology  of  potential  interest  in  this 
application  is  platinum  silicide  (PtSi).  This  technology  is  based  on  the 
formation  of  Schottky  barrier  diodes  through  the  deposition  of  a  thin  film  of 
metallic  platinum  on  silicon  Charge  Coupled  Device  (CCD)  or  Charge  Injection 
Device  (CID)  structures.  This  process  forms  a  silicide  layer  only  a  couple  of 
nanometers  thick.  The  resultant  IR  detector  devices  are  easily  fabricated  into 
large  arrays  based  on  well  developed  silicon  electronics  and  solid  state  imager 
technology.  PtSi  detectors  have  inherently  low  quantum  efficiencies;  however, 
this  drawback  is  usually  more  than  offset  by  excellent  fixed  pattern  noise 
characteristics.  For  applications  where  insufficient  integration  times  are 
available  (detection  takes  place  through  photo -electron  charge  accumulation) , 
PtSi  detectors  can  compete  with  other  technologies.  PtSi  focal  planes  are 
operated  at  a  temperature  of  77°K  to  freeze  out  dark  current. 
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ABSTRACT 

MMW  ISAR  imagery  and  IR  images  are  combined  to  yield  a  3D  point  scatterer  model  of  an  armored  vehicle. 
It  is  observed  that  colocated  MMW  and  IR  sensors  yield  orthogonal  views  of  a  target.  This  is  due  to  the 
fact  that  the  MMW  sensor  resolves  the  target  along  the  line-of-sight  (LOS)  and  the  IR  sensor  resolves  the 
target  orthogonal  to  the  LOS.  Consequently,  if  MMW  scatterers  can  be  identified  with  points  in  an  IR  image 
taken  from  the  same  viewpoint  then  the  3D  coordinates  of  the  scattering  centers  can  be  calculated.  In  this 
work  the  correspondence  between  points  in  the  MMW  ISAR  and  IR  images  is  achieved  by  using  a  second 
IR  viewpoint  upon  which  the  MMW  imagery  can  be  directly  overlayed.  A  3D  point  scatterer  model  of  an 
armored  vehicle  is  calculated  using  the  method  outlined  above.  Figures  are  presented  showing  several  views 
of  the  3D  model  obtained. 


1.  INTRODUCTION 

Millimeter  wave/infra-red  (MMW/IR)  sensor  fusion  addresses  the  problem  of  extracting  the  most  useful  target 
identification  information  from  dual  mode  data.  Fusion  of  information  can  be  attempted  at  various  levels.  The 
MMW  and  IR  information  can  be  processed  separately  yielding  a  MMW  target  identification  (ID)  and  a  seperate 
IR  target  ID.  Then  the  fusion  problem  is  reduced  to  resolving  conflicts  between  the  two  ID’s.  A  more  challenging 
approach  is  to  fuse  the  sensor  data,  i.e.,  combine  the  information  from  the  two  sensors  to  obtain  target  signature 
information  which  cannot  be  obtained  from  either  sensor  independently. 


The  data  to  be  analyzed  in  this  report  is  two  dimensional  imagery  derived  from  turntable  measurements.  IR 
imagery  and  MMW  images  formed  by  Inverse  Synthetic  Aperture  Radar  (ISAR)  processing  are  studied.  The  data 
used  to  form  the  images  was  taken  simultaneously  with  colocated  MMW  and  IR  sensors. 


Fusion  cannot  occur  without  some  association  or  registration  of  pixels  between  the  images  formed  from  the  two 
sensors.  While  studying  this  problem  we  realized  that  if  this  association  can  be  made  then  the  location  of  the  pixels 
in  three  dimensions  can  be  determined.  In  the  sections  that  follow  we  show  how  this  association  is  accomplished 


167 


for  turntable  measurement  data.  Using  this  information  a  three  dimensional,  polarimetric  model  of  target  radar 
scatterers  is  obtained  directly  from  sensor  measurement  data.  A  very  simple  analysis  of  the  point  scatterer  model 
yields  geometrical  information  about  the  vehicle  which  is  unattainable  by  a  similar  analysis  of  the  data  from  the  two 
sensors  done  independently  and  then  fusing  results. 

In  section  2  the  spatial  relationship  between  IR  and  MMW  ISAR  imagery  is  discussed.  A  discussion  of  MMW 
image  formation  is  presented  in  section  3.  A  detailed  description  of  how  MMW  point  scatterers  are  registered  onto 
IR  imagery  is  presented  in  section  4  followed  by  a  presentation  of  results  in  section  5. 

2.  INTUITIVE  VIEWPOINT 

We  define  the  intuitive  viewpoint  of  an  object  represented  by  an  image  to  be  the  direction  which  is  orthogonal 
to  the  plane  in  which  the  object  is  spatially  resolved.  This  determines  the  direction  of  the  intuitive  viewpoint  up  to 
a  sign.  The  sign  is  chosen  such  that  the  intuitive  viewpoint  has  a  positive  projection  on  the  LOS  or  such  that  it  is 
directed  into  the  ground  plane,  assuming  a  ground  plane  exists. 

For  visual  or  IR  imagery  the  intuitive  viewpoint  is  the  LOS.  For  ISAR  imagery  the  intuitive  viewpoint  is  orthogonal 
to  the  LOS.  Consequently,  colocated  MMW  and  IR  sensors,  even  though  they  possess  the  same  LOS  to  an  object, 
have  intuitive  viewpoints  which  are  orthogonal.  The  transformation  required  to  register  IR  and  MMW  images  depends 
on  the  relationship  between  their  respective  intuitive  viewpoints,  not  their  LOS’s.  Due  to  this  fact,  MMW  ISAR  and 
IR  imagery  taken  by  colocated  sensors  (same  LOS,  orthogonal  intuitive  viewpoints)  cannot  be  registered  by  simple 
scaling  and  translation  of  the  images.  The  only  exception  to  this  rule  is  the  case  where  the  resolution  of  the  target 
is  such  that  the  target  geometry  can  be  assumed  to  be  planar. 

An  interesting  consequence  of  the  orthogonality  of  the  MMW  ISAR  and  IR  intuitive  viewpoints  is  that  three 
dimensional  imaging  of  an  object  is  possible  from  a  single  viewpoint.  If  one  can  associate  scattering  centers  in  a 
MMW  ISAR  image  with  points  in  an  IR  image  from  the  same  viewpoint  then  a  correspondence  is  determined  between 
points  on  images  which  have  orthogonal  intuitive  viewpoints.  The  three-dimensional  coordinates  of  the  points  are 
determined  by  stereo  processing  the  imagery  for  the  two  orthogonal  intuitive  viewpoints.  Common  to  all  stereo 
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processing,  the  main  problem  is  the  determination  of  the  correspondence  between  points  in  the  two  images. 


3.  MMW  POINT  SCATTERER  IMAGE  FORMATION 


The  MMW  data  used  in  this  report  was  obtained  from  the  TABILS151  data  base.  The  data  is  fully  polarimetric 
turntable  measurement  data  taken  with  a  35  GHz,  stepped  frequency,  pulsed  radar.  Four  transmit/receive  states 
provide  fully  polarimetric  signatures. 

MMW  point  scatterer  images  are  formed  in  three  steps.  First,  each  transmit/receive  channel  is  converted  into 
a  two  dimensional  image  by  ISAR2  processing.  Each  pixel  of  the  four  image  planes  is  complex  valued;  the  relative 
phase  between  channels  is  important  for  the  polarimetric  processing  which  occurs  in  step  three.  These  ISAR  images 
suffer  from  artifacts  which  are  a  direct  result  of  the  Fourier  transform  processing  necessary  to  achieve  high  range  and 
cross-range  resolution.  The  effect  of  these  artifacts  is  to  blur  point  scatterer  responses. 

The  second  step  of  the  MMW  processing  is  the  removal  of  the  ISAR  artifacts.  We  employ  a  method  introduced  by 
Cameron3  termed  Point  Scatterer  Extraction  (PSE).  PSE  images  of  armored  vehicles  exhibit  distinct  point  scatterer 
responses.  The  intensity  of  a  point  scatterer  response  in  each  of  the  four  image  planes  depends  on  the  scatterer  type 
and  its  orientation  in  the  plane  of  the  LOS.  A  discussion  of  the  variability  of  scatterer  response  due  to  scatterer  type 
and  orientation  is  given  by  Huynen4 . 


The  third  step  in  the  MMW  image  formation  is  to  combine  the  responses  from  the  four  complex  PSE  images  into 
onetimage  and  to  remove  the  variability  of  scatterer  intensity  due  to  scatterer  type  and  orientation.  To  this  end,  a 
polarization  scattering  matrix  is  calculated  for  each  pixel  by  combining  the  responses  of  corresponding  pixels  in  the 
four  complex  PSE  images.  For  each  pixel,  the  scattering  matrix  is  decomposed  yielding  the  intensity  of  each  of  eleven 
basic  scatterer  types5.  Each  pixel  is  assigned  the  intensity  and  type  of  the  dominant  scatterer  in  its  polarimetric 


“Target  and  Background  Information  Library  System  (TABILS)  Users  Manual”,  Air  Force  Armament  Laboratory,  Air  Force 

1  Systems  Command.  United  States  Air  Force,  Eglin  Air  Force  Base,  Florida,  August,  1985. 

2  G.W.  Stimson,  Introduction  to  Airborne  Radar  ,  pp.  575-576,  Hughes  Aircraft  Co.,  El  Segundo,  Cal.,  (1983). 
W.L.  Cameron,  "Enhancement  of  Point  Scatterer  Responses  in  ISAR  Images",  Proceedings  of  the  GACIAC  Polarimetric 

3  Technology  Workshop,  Redstone  Arsenal,  Alabama,  Aug.  16-18.  1988. 

P.L.  Uslenghi  (Ed.),  J.R.  Huynen  (Auth.),  “Phenomenological  Theory  of  Radar  Targets".  Electromagnetic 

4  Scattering  ,  Academic  Press,  New  York,  1978. 

5  W.L.  Cameron  and  L.  Leung, “Feature  Motivated  Polarization  Scattering  Matrix  Decomposition",  submitted  to  RADAR  90. 
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response.  The  eleven  basic  scatterer  types  are:  non-reciprocal,  left  helix,  right  helix,  asymmetric,  trihedral,  cylinder, 
dipole,  narrow  diplane,  diplane,  quarter-wave  device,  and  symmetric. 

The  determination  of  the  dominant  scatterer  type  proceeds  as  follows.  Scatterers  are  divided  into  three  groups 
based  on  the  regularity  of  their  polarimetric  signatures.  The  three  groups  are:  non-reciprocal,  asymmetric  and 
symmetric.  Non-reciprocal  scatterers  are  scatterers  which  violate  the  reciprocity  principle6  ■ 7.  In  general,  non- 
reciprocal  responses  indicate  low  signal-to-noise  or  the  presence  of  special  materials.  Scatterers  which  obey  the 
reciprocity  principle  are  either  symmetric  or  asymmetric  scatterers.  Symmetric  scatterers  possess  an  axis  of  symmetry 
contained  in  the  plane  orthogonal  to  the  LOS.  Only  symmetric  scatterers  are  assigned  an  orientation,  which  is  the 
orientation  of  their  axis  of  symmetry  in  the  plane  orthogonal  to  the  LOS.  The  most  extreme  examples  of  asymmetric 
scatterers  Me  left  and  right  helices.  For  this  reason  left  and  right  helices  are  distinguished  from  other  asymmetric 
scatterers.  Scatterers  which  exhibit  asymmetric  responses  are  declared  to  be  left  or  right  helices  if  their  helical 
response  is  great  enough,  otherwise  they  are  simply  declared  to  be  asymmetric.  Symmetric  scatterers  are  declared  to 
be  one  of  the  following  types:  trihedral,  cylinder,  dipole,  narrow  diplane,  diplane,  quarter-wave  device,  or  symmetric 
(non-specific  symmetric  scatterer). 

The  final  MMW  image  has  three  pieces  of  information  associated  with  each  pixel:  scatterer  intensity,  scatterer 
type,  scatterer  orientation.  The  final  images  are  simple  enough  that  individual  scatterers  can  be  matched  to  features 
observed  in  IR  images. 


4,  CORRESPONDENCE  FOR  STEREO  PROCESSING 


As  we  are  analyzing  turntable  measurement  data  the  correspondence  problem  can  be  solved  simply  by  using  two 
IR  images.  The  first  IR  image  is  chosen  such  that  its  LOS  is  orthogonal  to  the  MMW  LOS.  This  IR  image  will  have 
an  intuitive  viewpoint  equal  to  that  of  the  MMW  image.  The  other  IR  image  is  chosen  with  the  same  LOS  as  the 
MMW  image.  In  this  case  the  intuitive  viewpoints  will  be  orthogonal.  The  IR  and  MMW  images  with  equal  intuitive 
viewpoints  can  be  registered  by  a  simple  scaling  and  translation  of  the  images.  Registration  of  these  images  establishes 

6  D.S.  Saxon,  “Tensor  Scattering  Matrix  for  the  Electromagnetic  Field”,  Phys.  Rev.,  100 , 1771,  (1955). 

7  J.H.  Van  Vleck,  Theory  of  Electric  and  Magnetic  Susceptibilities  ,  Oxford  University  Press,  London,  (1932). 
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the  correspondence  between  MMW  scatterer  positions  and  IR  features.  Using  this  correspondence  individual  MMW 
scatterers  can  be  registered  onto  the  IR  image  which  has  the  intuitive  viewpoint  which  is  orthogonal  to  the  MMW 
intuitive  viewpoint,  i.e.,  the  same  LOS.  Using  this  procedure  the  MMW  scatterer  positions  are  determined  from  two 
orthogonal  viewpoints  and  the  three-dimensional  coordinates  of  the  MMW  scatterers  are  derived  by  simple  stereo 
processing. 

For  the  example  given  (see  Fig.  2)  we  have  chosen  to  use  MMW  data  with  an  LOS  corresponding  to  a  sensor 
depression  angle  of  40°  and  the  turntable  positioned  at  0°  (head-on  view  of  target).  The  MMW  intuitive  viewpoint 
is  orthogonad  to  this.  The  IR  image  which  has  the  same  intuitive  viewpoint  as  the  MMW  image  has  an  LOS 
corresponding  to  a  sensor  depression  angle  of  50s  with  the  turntable  rotated  180°. 

In  order  for  the  IR  and  MMW  images  to  be  aligned  by  a  simple  scale  chamge  and  translation  the  IR  image  must 
be  reflected  about  the  horizontal  and  verticad  axes.  The  reflections  would  not  be  necessary  if  the  repositioning  of  the 
viewpoint  were  accomplished  by  a  physical  rotation  of  the  sensor  by  90°  about  the  x  axis  (see  Fig.  1).  Note  that 
repositioning  the  sensor  in  this  way  causes  a  flip  in  the  vertical  direction  in  the  rest  frame  of  the  sensor  relative  to  the 
target  vertical.  Repositioning  the  viewpoint  using  a  180°  rotation  of  the  turntable  (or  sensor  about  the  z  axis)  does 
not  flip  the  verticad  of  the  sensor  relative  to  the  target,  but  it  does  flip  the  sensor’s  horizontad  orientation  relative  to 
the  target.  For  a  sensor  depression  angle  of  45°,  rotation  about  the  x  axis  can  be  simulated  by  rotating  180°  about 
the  z  axis  and  then  reflecting  the  image  about  the  verticad  and  horizontal  axes.  In  general,  if  the  sensor  depression 
angle  is  0,  the  same  process  applies  except  that  the  180°  rotation  data  must  have  a  depression  angle  of  90°  —  0. 

Figure  2  illustrates  the  fact  that  MMW  PSE  and  IR  images  which  have  the  same  intuitive  viewpoint  can  be 
registered  by  a  simple  scale  change  and  translation  of  the  images.  Figure  2a  is  the  PSE  image  of  an  armored  vehicle 
which  has  been  scaled  to  match  the  dimensions  of  the  IR  images,  Figs.  2b,  2c.  We  have  sacrificed  detail  for  clarity  by 
displaying  only  the  locations  of  the  major  scatterers  in  the  PSE  image,  omitting  their  intensity,  type  and  orientation. 
Figure  2b  is  an  IR  image  of  the  vehicle  which  has  the  same  LOS  as  the  PSE  image  (  40°  sensor  elevation  angle,  0° 
turntable  rotation).  Figure  2c  is  an  IR  image  of  the  vehicle  which  has  (approximately)  the  same  intuitive  viewpoint  as 
the  PSE  image  (IR  LOS  has  40°  sensor  elevation  angle,  180°  turntable  rotation  angle).  Figure  2c  has  been  reflected 
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horizontally  and  vertically  to  compensate  for  axis  sign  reversal  as  discussed  previously. 

The  scaling  of  both  the  IR  and  MMW  PSE  images  is  known.  The  only  difficulty  in  registering  the  images  with 
the  same  intuitive  viewpoint  is  in  determining  the  relative  translation.  The  PSE  image  can  be  decomposed  into  four 
recognizable  sets  of  points.  The  first  set  is  composed  of  the  points  which  lie  roughly  in  two  straight  lines  along  either 
side  of  the  vehicle.  The  second  set  of  points  are  the  two  symmetrically  placed  returns  from  the  headlights  at  the 
front  of  the  vehicle.  The  remaining  two  sets  of  points  are  two  clusters.  One  cluster  lies  just  above  the  headlight  set 
and  is  due  to  scattering  from  the  roof  of  the  vehicle.  The  other  set  of  points  is  approximately  in  the  shape  of  the 
letter  “V” .  The  apex  of  this  set  is  due  to  scattering  from  the  launcher  support  column  and  the  extremities  are  due 
to  scattering  from  the  launchers.  The  far  rear  of  the  vehicle  is  not  visible  in  the  PSE  image  causing  the  signature 
to  be  shorter  than  the  vehicle,  even  after  compensating  for  aspect  angle.  The  relative  translation  is  determined  by 
aligning  the  four  sets  of  points  with  their  respective  features  in  the  corresponding  IR  image.  This  is  not  possible  to 
accomplish  using  the  MMW  and  IR  images  with  the  same  LOS,  Figs.  2a  and  2b.  The  result  of  aligning  the  MMW 
and  IR  images  with  the  same  intuitive  viewpoint  is  illustrated  in  Fig.  2d. 

The  IR  image  features  underlying  the  MMW  scatterer  positions  in  Fig.  2d  are  identified  with  IR  features  from 
the  (nearly)  orthogonal  IR  viewpoint  image,  Fig.  2b,  by  direct  comparison  of  the  images.  The  scatterers  in  the 
MMW  image,  Fig.  2a,  are  indirectly  identified  with  positions  on  the  IR  image  with  the  same  LOS,  Fig.  2b,  using 
the  IR  image  (Fig.  2c)  with  the  same  intuitive  viewpoint  as  the  MMW  image  as  an  intermediary. 

5.  RESULTS 

Employing  the  method  described  in  section  4,  two  dimensional  scatterer  positions  are  determined  from  two 
orthogonal  viewpoints.  Define  the  primed  coordinate  system  such  that  j/  is  directed  anti-parallel  to  the  MMW 
sensor  LOS,  x'  is  parallel  to  the  x  axis  (see  Fig.  1)  and  choose  z'  such  that  it  completes  the  right-handed  coordinate 
system.  The  MMW  PSE  image,  Fig.  2a,  yields  (x',j/)  coordinates  of  scatterers  directly.  Indirect  registration  (as 
described  in  the  previous  section)  of  the  MMW  scatterer  positions  onto  Fig.  2b  yields  the  ( x',z ')  coordinates  of 
scatterers.  Combining  these  results  yields  the  (i',^,  z')  coordinates  of  the  MMW  scatterers. 
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Figures  3  and  4  present  various  views  of  a  3D  scatterer  model  derived  from  the  image  data  presented  in  Fig.  2. 
Fig.  3a  shows  scatterer  positions  obtained  directly  from  the  MMW  PSE  image  (Fig.  2a)  which  corresponds  to  an 
intuitive  viewpoint  which  is  orthogonal  to  the  MMW  LOS.  Scatterer  positions  are  indicated  by  cubes.  Fig.  3b  is  the 
model  viewed  from  the  positive  y  direction  (refer  to  Fig.  1  coordinate  system).  Fig.  3c  is  the  model  viewed  from 
directly  overhead.  The  contraction  of  the  apparent  length  of  the  target  due  to  aspect  angle  can  be  seen  by  comparing 
Figs.  3a  and  3c.  Figs.  4a  and  4b  are  views  of  the  model  from  broadside  and  askew,  respectively. 

MMW  scatterer  types,  as  well  as  intensity,  are  displayed  in  Figure  6.  The  symbols,  defined  in  Fig.  5,  are  registered 
directly  and  indirectly,  as  discussed  above,  onto  two  IR  views.  Figure  6a  is  the  (x'.y')  plane  view  and  Fig.  6b  is  the 
(a plane  view. 

Display  of  the  3D  model  with  scatterer  intensity,  orientation  and  type  requires  a  set  of  3D  symbols  to  be  defined. 
This  set  is  defined  in  Fig.  7.  Figure  8  shows  two  views  of  the  3D  scatterer  model  using  3D  symbols  to  indicate  scatterer 
type,  orientation  and  intensity.  Scatterer  intensities  are  indicated  by  scaling  the  scatterer  symbols  displayed  in  Fig. 
7. 

The  vehicle  length,  width  and  height  dimensions,  in  meters,  are  5.8  x  2.4  x  2.3.  Vehicle  dimensions  as  measured 
using  the  IR  and  MMW  PSE  images  independently  are  5.68  x  2.4  and  3.46  x  2.09,  respectively.  These  measurements 
do  not  account  for  sensor  aspect,  but  are  simply  boundary  measurements  of  blobs,  in  the  case  of  IR,  and  clusters,  in 
the  case  of  MMW.  These  are  measurements  one  might  expect  an  automatic  target  recognition  algorithm  to  provide. 
The  vehicle  dimensions  derived  from  the  3D  model  are  4.51  x  2.09  x  1.37. 

If  the  vehicle  is  modeled  as  a  two  dimensional  rectangle  in  the  plane  of  the  turntable  then  one  would  expect  a 
contraction  in  the  apparent  length  of  the  vehicle  due  to  aspect.  The  vehicle  height  tends  to  increase  its  apparent 
length.  For  the  case  given  these  twc  ,ects  cancel  each  other  yielding  an  IR  length  estimate  which  is  quite  accurate. 

The  MMW  length  measurement  is  shorter  than  one  would  expect,  even  taking  into  account  the  effect  of  aspect. 
This  is  due  to  shadowing  of  scatterers  in  the  rear  of  the  vehicle  as  is  apparent  from  Fig.  2d. 

The  3D  model  length,  width  and  height  measurements  were  made  along  the  vehicle  axes  and  thus  are  not  affected 
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by  aspect  angle.  The  main  deficiency  in  the  3D  model  measurements  stems  from  the  shadowing  of  scatterers  in  the 
rear  of  the  vehicle  and  underneath  the  lower  apron  of  the  vehicle.  The  height  of  the  vehicle  above  the  lower  apron 
is  approximately  .89  meters  which  is  somewhat  more  in  agreement  with  the  3D  model  measurement  than  the  total 
vehicle  height. 

The  method  presented  is  useful  for  unraveling  some  of  the  complexities  of  MMW  scattering,  as  well  as  under¬ 
standing  the  relationship  between  IR  and  MMW  signatures.  A  more  complete  understanding  of  target  geometry  is 
obtained  by  fusing  sensor  reports  before  features  are  extracted. 
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Figure  1.  Orientation  of  coordinate  system  fixed  in  sensor  relative 

to  target  coordinate  system  for  repositioning  of  the  sensor  by 
rotating  about:  a)  the  x  axis,  b)  the  z  axis.  Style  of  arrow 
indicates  vectors  in  the  plane  of  the  page  (A),  out  of  the 
page  (^)  or  into  the  page  (^). 
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Figure  2.  Images  of  armored  vehicle  derived  from  sensor  data:  a)  PSE  image  scat- 
terer  positions,  LOS  directed  40°from  ground  plane  and  head-on.  b)  IR 
image  with  LOS  same  as  (a),  c)  IR  image  with  LOS  nearly  orthogonal 
to  (a)  and  (b),  d)  overlay  of  (c)  with  (a). 
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Figure  5.  Scatterer  type  symbols. 


(a)  (b) 


Figure  6.  MMW  scatterer  symbols  registered  onto  IR  images:  a)  direct  registration 
by  scaling  and  translation,  b)  individual  scatterers  registered  using  3D 
model  position  information. 
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The  data  necessary  for  robust  target  recognition  algorithm  development  are  not  always  readily  available,  especially 
when  advanced  sensors  are  being  considered.  Data  collection  exercises  are  almost  always  plagued  by  high  cost, 
sensor  intermittence,  and  lack  of  terrain  and  target  variety.  As  a  result,  target  recognition  algorithms  developed  to 
date  suffer  from  homogeneous  testing  conditions.  An  example  of  an  advanced  sensor  under  consideration  for  target 
recognition  systems  is  an  imaging  laser  radar  sensor.  Laser  radars  can  provide  high  resolution  range  information  as 
well  as  high  resolution  reflectance  information  about  the  target  These  observables  are  invariant  under  conditions  of 
varying  target  temperature,  ambient  temperature,  and  lighting.  The  invariant  observables  and  the  3-D  geometric 
nature  of  the  data  make  laser  radar  a  good  sensor  choice  for  target  recognition  algorithm  development  In  order  to 
circumvent  the  data  availability  problems  that  exist  some  researchers  have  been  and  will  continue  to  utilize 
synthetic  laser  radar  imagery  for  recognition  algorithm  development. 

In  the  Laser  Radar  ATR  Algorithms  (LARAA)  program,  we  are  currently  developing  a  synthetic  laser  radar  image 
generation  capability  that  will  be  somewhat  automated,  incorporate  a  laser  radar  sensor  model,  and  will  consist  of  a 
variety  of  targets,  clutter,  and  terrain  selections.  This  capability  will  utilize  currently  available  geometric  target 
models  such  as  the  Ballistic  Research  Labs  CSG  models  and  the  ERIM  wireframe  models,  and  will  integrate  the 
geometric  target  models  with  the  terrain,  clutter,  and  sensor  models  developed  in  the  program.  The  result  will  be  a 
extensible  and  modular  synthetic  image  generation  capability  and  a  large  database  of  synthetic  laser  radar  range 
imagery  for  use  in  recognition  algorithm  development 

In  this  paper  we  will  describe  the  synthetic  laser  radar  image  generation  effort  in  the  LARAA  program  including  an 
overview  of  the  geometric  target  models,  terrain  models,  sensor  model,  and  the  overall  integration  process. 
Preliminary  synthetic  imagery  will  also  be  presented. 


I.  INTRODUCTION 


The  development  of  electronic  terrain  board  technology  is  a  rapidly  growing  area  of  research.  There  are  several 
different  applications  for  this  technology;  some  of  these  applications  are  listed  below: 
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•  battlefield  simulation  for  weapon  systems  testing 

•  flight  training  (man  in  the  loop) 

•  sensor  and  target  phenomenology  modeling 

•  target  recognition  algorithm  development  and  testing 

There  are  two  major  areas  of  technology  development  in  electronic  terrain  board  research.  One  area  is  the 
development  of  techniques  for  data  compression/representation  and  computational  expedience  to  solve  problems 
related  to  the  real  time  generation/presentation  of  sequences  of  data.  The  other  area  is  the  understanding 
(quantification)  of  object/sensor  phenomenology  in  order  to  solve  problems  related  to  the  generation  of  accurate  and 
verified  imagery  for  a  particular  sensor.  The  work  described  in  this  paper  fits  into  the  latter  area  of  technology 
development 

A  controlled  environment  (input  data  set)  is  an  essential  need  in  the  development  and  testing  of  target  recognition 
algorithms,  especially  if  algorithm  performance  understanding  is  an  objective.  The  data  necessary  for  robust  target 
recognition  algorithm  development  are  not  always  readily  available,  especially  when  advanced  sensors  (i.e.  laser 
radar)  are  being  considered.  Data  collection  exercises  are  almost  always  plagued  by  high  cost,  sensor  intermittence, 
and  lack  of  terrain  and  target  variety.  As  a  result,  target  recognition  algorithms  developed  to  date  suffer  from 
homogeneous  testing  conditions.  The  purpose  of  this  paper  is  to  describe  on-goim:  (Torts  in  the  development  of  a 
synthetic  laser  radar  image  generation  capability  that  will  provide  a  controlled  environment  for  recognition 
algorithm  development. 

The  work  presented  here  is  currently  being  performed  under  the  CCNVEO/Honeywell  Laser  Radar  ATR  Algorithm 
(LARAA)  program.  The  objective  of  the  Synthetic  Laser  Radar  Image  Generation  task  in  LARAA  is  to  circumvent 
the  data  availability  problems  that  exist  by  developing  a  data  synthesis  capability  that  will  augment  "real"  data  in 
target  recognition  algorithm  development  efforts.  Section  II  will  present  an  oven  u  «  .»f  the  technical  approach. 
Section  III  will  present  some  concluding  remarks. 
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n.  TECHNICAL  APPROACH 


In  order  to  avoid  duplication  of  previous  efforts,  our  technical  approach  for  synthetic  laser  radar  image  generation 
borrows  from  previously  developed  "pieces"  of  modeling  technology.  The  pieces  of  technology  that  have  not  been 
fully  developed  will  be  enhanced  as  necessary,  and  the  overall  capability  will  be  integrated  into  a  flexible  and 
expandable  synthetic  image  generation  capability.  The  six  major  "pieces"  of  technology  needed  for  synthetic  image 
generation  are  as  follows: 

1 .  Geometric  target  models 

2.  Atmospheric  propagation/scattering  models 

3.  Laser  radar  sensor  model 

4.  Terrain  models 

5.  Clutter  models 

6.  Model  integration  executive 

For  this  task  in  the  LARAA  program,  we  are  "bootstrapping"  off  of  relevant  research/development  in  4  of  the  6 
listed  technology  areas.  The  clutter  model  and  the  model  integration  executive  are  the  technology  areas  that  will  be 
developed  "from  scratch"  in  the  LARAA  program  in  order  to  meet  the  specific  needs  of  synthetic  image  generation 
for  future  ATR  algorithm  development 

In  the  remainder  of  this  section,  an  overview  of  the  LARAA  program  technical  approach  for  each  of  the  6  technology 
areas  listed  above  is  presented. 


Geometric  Target  Models 

The  Ballistic  Research  Laboratory  (BRL)  has  developed  an  extensive  CAD  solid  modeling  system  and  ray-tracing 
benchmark.  Many  different  targets  are  available  from  BRL  and  the  documentation  and  utility  of  the  software  is 
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excellent.  Visible  (E-0)  target  signatures  can  be  rendered  with  various  lighting  models  with  the  baseline  software; 
other  available  software  can  render  thermal  signatures  or  radar  signatures  of  the  targets.  LARAA  will  develop 


software  (using  the  BRL  CAD  tools  where  appropriate)  to  generate  high  sampling  resolution  range  signatures  of  the 
targets. 


The  Environmental  Institute  of  Michigan  (ERIM)  has  developed  wireframe  target  models  for  ATR  algorithm 
evaluation  in  the  PAIRSTECH  program.  These  models  are  much  less  detailed  than  the  BRL  solid  models,  but  may 
prove  to  be  adequate  for  some  synthetic  imagery  (particularly  low  spatial  resolution  imagery).  Under  the  LARAA 
program,  we  will  develop  range  signature  rendering  software  for  the  wireframe  models.  A  qualitative  and  quantitative 
benchmark  comparing  the  ERIM  wireframe  and  the  BRL  CSG  models  will  be  performed  in  order  to  aid  in  the 
selection  of  target  models  for  the  synthetic  imagery. 


Several  different  software  packages  exist  that  are  complete  atmospheric  propagation  models,  handling  broadband 
and/or  narrowband  electromagnetic  energy.  Because  software  packages  are  available  that  are  directly  applicable,  the 
LARAA  program  has  no  planned  upgrades  to  this  technology.  Rather,  LARAA  will  (with  lowest  priority)  integrate 
atmospheric  effects  into  the  synthetic  image  generation  capability  using  one  or  more  or  the  available  software 
packages.  This  is  considered  to  be  a  low  priority  item  for  this  phase  of  the  LARAA  program  so,  as  a  baseline,  one 
set  of  atmospheric  degradations  for  nominal  atmospheric  conditions  will  be  used.  Software  "hooks"  will  be  left  the 
future  integration  of  a  complete  atmospheric  model. 


The  laser  radar  sensor  model  is  one  of  the  most  important  factors  in  creating  realistic  synthetic  imagery.  The  model 
for  an  imaging  laser  radar  sensor  can  be  broken  up  into  two  parts  -  the  scanning  (or  imaging)  mechanism  for  the 
sensor  and  the  physics  (phenomenology)  associated  with  the  transmission,  scattering,  and  detection  of  the  laser 
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C  and  E  are  arbitrary  functions  useful  for  modifying  the  amplitude  and  shape  of  the  gaussian  surface,  and  x’  and  y'  are 
points  (x,y)  after  applying  a  rotation  to  them: 

x'  =  xcos  (8)  -  ysin  (0) 
y'  =  xsin  (8)  +  ycos  (0) 

0  is  the  rotation  angle  of  the  guassian  surface  in  the  xy  plane. 

The  elevation  z  of  a  given  (x,y)  coordinate  is  the  contribution  of  all  the  m  gaussian  surfaces  at  that  point: 

m 

z(x,y)  =  Z  Gi(x,y) 
i- 1 


For  a  given  gaussian  surface  G,  seven  parameters  control  its  shape:  0,  <yx,  <yy,  px>  |iy,  E,  and  C.  Including  m,  the 
number  of  gaussian  surfaces  that  constitute  the  terrain,  these  variables  can  create  terrains  of  arbitrary  shape  and 
complexity. 

Once  the  terrain  is  defined  in  this  fashion,  it  can  be  raytraced  to  acquire  the  range  map.  Figure  1  illustrates  a  terrain 
map  and  the  resulting  relative  range,  absolute  range,  and  intensity  maps. 

Since  both  terrain-generation  methods  create  locally  smooth  surfaces,  fractal  perturbations  can  be  incorporated  into 
the  terrain  model  to  provide  local  roughness  and  add  "landscape"  features  to  the  terrain  (trees,  treelines,  etc.). 

Clutter  models 

Another  issue  this  program  addresses  is  clutter  modeling.  For  our  purposes,  a  clutter  object  is  defined  as  an  object 
that  shares  a  certain  degree  of  similarity  with  a  given  target  that  an  ATR  intends  to  recognize.  This  degree  of 
similarity  can  be  parameterized  as  the  difficulty  of  the  clutter  object.  This  clutter  parameterization  will  control  the 
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beam.  The  scanning  model  describes  the  spatial  sampling  that  the  laser  radar  will  perform;  the  phenomenological 
model  describes  the  target/laser  beam  interaction,  signal-to-noise  ratio,  range  measurement  errors  and  dropouts.  We 
are  applying  previous  work  that  has  developed  mathematical  models  for  particular  laser  radar  systems  in  the  creation 
of  an  imaging  laser  radar  sensor  model  that  is  more  accurate  than  heuristically  "adding  gaussian  noise"  to  the  image 
or  "smoothing"  the  image. 


Terrain  models 

Two  methods  for  creating  elevation  data  for  the  terrain  model  are  being  exploited;  digital  elevation  data  developed  by 
the  DMA  (Defense  Mapping  Agency),  NCIC  (National  Cartographic  Information  Center),  and  CCNVEO,  and  by 
using  gaussian  surfaces. 

The  elevation  data  obtained  from  the  DMA  and  NCIC  represents  coarse  samples  of  real  terrain.  The  resolution 
(distance  between  samples)  of  these  data  varies  between  approximately  10  and  40  meters.  The  CCNVEO  data 
provided  are  the  elevation  map  of  CCNVEO's  physical  terrain  board.  This  fictitious  terrain  is  divided  into  1 
kilometer  square  sectors  which  are  subdivided  into  quadrants  and  then  into  rows.  Each  row  consists  of  five  elevation 
points  that  are  two-hundred  meters  apart  As  a  result  of  the  low  resolution  of  these  data,  the  rendered  terrain  will  be 
quite  smooth. 

Combinations  of  gaussian  surfaces  are  suitable  for  creating  more  complicated  and  arbitrary  terrains  than  the  digital 
elevation  data.  The  gaussian  surfaces  are  also  well  suited  to  automatic  synthetic  image  generation,  since  the  surfaces 
can  be  easily  created  by  using  random  parameters  (perhaps  sampled  from  a  given  density).  The  form  of  the  gaussian 
equation  has  been  modified  for  this  application: 

0  -(*' '  Hx)2  (y'-liy)2  , 

C(x  v)  9  +  9  1 

G(x,y)  =  <*xz  <*yz  +  E(x,y) 


188 


automatic  generation  of  the  synthetic  laser  radar  imagery.  Each  image  is  created  based  on  the  a  priori  clutter 
difficulty  selected  for  that  image.  Thus  the  performance  of  a  recognition  algorithm  can  be  evaluated  as  a  function  of 
clutter  difficulty. 

Clutter  will  be  modeled  as  perturbations  of  targets.  For  example,  a  clutter  object  for  an  M60  tank  could  be  the  tank 
itself  with  the  turret  removed.  Since  some  targets  exhibit  local  similarities,  one  target  can  be  perturbed  to  be  a 
clutter  object  for  another  target.  Other  perturbation  methods  include  moving  subsets  of  a  target  model’s  vertices, 
slicing  a  target  model  with  a  given  function,  adding  artifacts  to  target  surfaces,  etc.  Eventually  the  types  of 
perturbations  formed  will  be  dependent  on  the  level  of  clutter  difficulty  desired.  For  example,  clutter  formed  as  a 
result  of  perturbing  a  facet  of  a  target  model  may  cause  a  recognition  algorithm  more  problems  than  clutter  formed 
by  removing  the  front  half  of  the  target  model.  The  minimum  Hausdorff  distance  *  between  the  target  model  and  the 
clutter  object  will  aid  the  determination  of  the  relationship  between  clutter  difficulty  and  perturbation  type. 

Model  Integration 

The  model  integration  executive  will  integrate  each  piece  of  technology  into  an  comprehensive  synthetic  image 
generation  capability.  The  integration  approach  will  be  designed  to  maintain  flexibility  so  that  upgrades  and  changes 
to  each  technology  nugget  can  be  easily  accommodated. 

The  overall  approach  to  synthetic  laser  radar  image  generation  is  illustrated  in  Figure  2.  The  approach  calls  for 
having  a  database  available  for  each  element  of  the  desired  synthetic  scene  (e.g.  the  targets,  terrain,  and  clutter).  The 
database  for  each  scene  element  will  contain  the  high  resolution  range  map  generated  by  the  ray  tracing  operation  (or 
any  other  suitable  method)  and  will  contain  the  reflectance  distribution  statistics  for  that  particular  element.  The 
desired  synthetic  scene  is  specified  by  the  scene  description  files,  which  can  be  created  and  edited  interactively  using 
an  overlay  display  tool  on  the  Sun  computer,  giving  visual  feedback  on  what  the  scene  will  look  like.  The  clutter 

1  D.  Mumford,  "The  Problem  of  Robust  Shape  Description",  Center  for  Intelligent  Control  Systems,  Harvard 
University,  Cambridge,  Mass.,  1987. 
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elements  may  be  treated  separately  in  order  to  randomize  its  placement  and  incorporate  parameters  that  describe  the 
difficulty  of  the  clutter;  this  cannot  be  determined  until  the  clutter  models  we  will  use  are  developed  further. 

Some  synthetic  imagery  generated  to  date  will  be  presented  at  the  symposium.  Because  of  the  difficulty  of  printing 
color  results,  the  synthetic  imagery  are  not  included  in  the  written  paper. 

ffl.  SUMMARY 

In  this  paper,  we  have  presented  our  ongoing  efforts  in  the  areas  of  synthetic  laser  radar  image  generation.  Our 
approach  will  provide  a  means  for  generating  parameterized  synthetic  laser  radar  imagery  for  use  in  recognition 
algorithm  research.  The  use  of  a  controlled  algorithm  development  and  evaluation  environment  will  result  in  a  better 
understanding  of  algorithm  performance  and  ultimately  in  more  robust  recognition  algorithms. 
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a.  Terrain  map.  o.  intensity,  c.  Relative  range,  a.  Absolute  Range 


Figure  1.  Synthetic  Terrain  Map  Results 


Figure  2.  Synthetic  Laser  Radar  Image  Generation  Approach 
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ABSTRACT 

The  IMTURB  model  characterizes  the  effects  of  clear-air  optical 
turbulence  on  electro-optical  (EO)  imaging  devices  and  laser  trans¬ 
mitters  operating  in  the  atmospheric  surface  boundary  layer  and  is 
valid  for  wavelengths  from  the  visible  through  the  far  infrared. 
Similarity  theory  and  the  Kolmogorov  principle  of  universal  equi¬ 
librium  are  employed  to  characterize  optical  turbulence  structure 
over  regions  typical  of  applications ,  based  on  simple  environmental 
observables.  A  profile  for  the  refractive  index  structure  parameter 
is  estimated,  then  used  as  input  to  a  weak  perturbation  propagation 
model.  Receiver  coherence  diameter,  log-amplitude  variance,  scin¬ 
tillation  averaging  length,  isoplanatism  effective  path  length,  and 
related  subsidiary  propagation  statistics  are  then  estimated  for  a 
selected  propagation  path.  Outer  and  inner  scale  profiles  are  also 
estimated  for  advanced  applications.  Model  results  for  two  sample 
cases  are  discussed. 


1.0  INTRODUCTION 

It  is  well  understood  that  irregularities  in  the  atmospheric  refractive 
index  result  in  distortion  of  propagating  electromagnetic  waveforms.  In  a  land¬ 
mark  analysis  based  on  the  work  of  Kolmogorov,1  2  3  it  was  postulated  that 
a  primary  source  of  such  irregularities  is  the  turbulent  motion  of  the 

This  paper  was  originally  presented  at  the  meeting  of  AGARD,  Copenhagan,  Denmark 
(1989). 
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atmosphere  in  the  presence  of  temperature  and  moisture  gradients.  It  was  fur¬ 
ther  postulated  that  these  irregularities  possess  a  quantifiable  statistical 
structure.4  The  validity  of  these  postulates  is  generally  accepted,  and  the 
term  "optical  turbulence"  has  come  into  general  usage  to  denote  random, 
turbulence- induced  fluctuations  in  the  atmospheric  refractive  index. 

When  an  electromagnetic  wave  propagates  through  an  atmosphere  where  optical 
turbulence  is  present,  the  wave  front  acquires  a  progressive  degree  of  distor¬ 
tion.  Departure  of  the  phase  front  from  its  nominal  shape  results  in  a  decrease 
in  registered  image  quality  for  electro-optical  (EO)  imaging  systems.  Images 
appear  distorted  and  blurred.  A  laser  beam  exhibits  an  increase  in  beam 
diameter  in  excess  of  that  predicted  by  diffraction  theory.  In  addition,  the 
entire  beam  is  subject  to  optical  turbulence- induced  random  translations  of  the 
beam  centroid,  a  phenomenon  called  beam  wander.  The  purpose  of  the  IMTURB  model 
is  to  quantify  these  and  related  effects.5 

IMTURB  differs  from  typical  propagation  models  for  optical  turbulence  in 

that  a  profile  is  calculated  for  the  refractive  index  structure  parameter  (often 

2 

referred  to  as  a  profile)  from  simple  meteorological  and  environmental 
information.  Previous  methods  often  relied  on  the  use  of  one  or  more  measured 
point  values  for  the  refractive  index  structure  parameter.  From  these  measure¬ 
ments,  profiles  were  extrapolated  by  a  scaling  law.6  In  the  region  of  the 
atmosphere  where  terrain  surface  features  play  a  significant  role  in  determining 
optical  turbulence  structure,  such  an  approach  may  impose  severe  limitations. 
This  region  of  the  atmosphere,  called  the  surface  boundary  layer,  ranges  in 
height  from  tens  to  hundreds  of  meters  depending  on  atmospheric  conditions. 
Above  the  surface  boundary  layer,  the  degree  of  statistical  consistency  over 
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regions  commensurate  with  propagation  path  length  may  be  such  that  point 

2 

measurements  of  or  even  free  atmosphere  models  may  be  adequate  to  specify 
global  optical  turbulence  structure.  This  is  the  case  with  many  astronomical 
applications.  However,  within  the  surface  boundary  layer,  a  propagation  path 
may  cross  several  differing  optical  turbulence  regimes,  each  driven  by  slightly 
different  environmental  factors.  Point  measurements,  extended  to  a  profile  with 
a  power  law  and  extrapolated  over  the  total  propagation  path,  may  give  rise  to 
misleading  results.  In  response  to  the  problems  inherent  in  modeling  this  sce¬ 
nario,  IMTURB  employs  environmental  parameters  exhibiting  statistical  consis¬ 
tency  over  regions  consistent  with  most  laser  transmitter  and  imaging 
applications . 

In  the  complex  environment  of  the  atmospheric  surface  boundary  layer,  a 
precise  and  comprehensive  description  of  surface  energy  fluxes  driving  the  opti¬ 
cal  turbulence  mechanism  is  difficult.  The  number  of  observations  necessary  to 
quantify  point-specific  behavior  of  the  surface  fluxes  is  greater  than  available 
even  in  we  11 -instrumented  field  experiments.  In  addition,  spatial  and  temporal 
variations  in  environmental  factors  along  the  length  of  the  propagation  path 
challenge  the  validity  of  even  the  most  carefully  performed  site  characteriza¬ 
tions.  To  combat  these  difficulties,  the  modeling  basis  for  IMTURB  combines  the 
micrometeorology  of  the  atmospheric  surface  boundary  layer  with  the  well- 
understood  and  documented  weak  fluctuation  theory  of  electromagnetic  propagation 
through  optical  turbulence.7  8  9  1 0  To  characterize  the  micrometeorology, 
Obukhov  similarity  theory11  12  13  14  is  used  to  determine  basic  flux  profile 
relationships.  The  Kolmogorov  principle  of  universal  equilibrium1 5  employs 
these  relationships  to  define  profiles  for  the  turbulent  inner  and  outer  scales 
and  the  refractive  index  structure  parameter.  Both  Obukhov  similarity  theory 
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and  the  Kolmogorov  principle  of  universal  equilibrium  are  concerned  with  model¬ 
ing  the  behavior  of  surface  energy  fluxes  in  a  statistically  consistent  environ- 

2 

ment.  Therefore,  the  shape  of  the  calculated  Cn  profile  reflects  the  total 
environmental  state  more  completely  than  would  a  profile  obtained  from  a  scaling 
law.  A  profile  for  outer  scale  occurs  naturally  in  the  application  of  the  uni¬ 
versal  equilibrium  principle;  inner  scale  is  obtained  using  a  method  described 
by  Ochs  and  Hill.18  Although  the  version  of  IMTURB  currently  available  through 
the  Electro-Optical  Systems  Atmospheric  Effects  Library  (EOSAEL)  uses  the 
Kolmogorov  spectrum  to  calculate  propagation  statistics,  planned  updates  include 
extension  to  strong  perturbation  theory  and  use  of  the  modified  von  Karman,17 
Hill,18  or  Andrews  spectrums . 19  Profiles  for  inner  and  outer  turbulence 
scales  are  included  for  advanced  applications. 

2.0  MODELING  OPTICAL  TURBULENCE  STRUCTURE 

Obukhov  similarity  theory  is  used  to  calculate  surface  energy  fluxes  and 

related  quantities  for  a  specified  geographical  location,  date,  and  time.  The 

meteorological  observables  required  are  percent  cloud  cover  and  a  value  for 

temperature,  pressure,  and  windspeed.  The  earth's  aerodynamic  features  are 

summarized  by  specification  of  a  roughness  length20  or  by  calculation  of  the 

roughness  length  from  an  observed  roughness  element.21  The  algorithms  used  by 

IMTURB  to  estimate  flux  profile  relationships  were  first  proposed  by  Stewart22 

for  use  in  determining  turbulent  velocity  spectrums  over  complex  terrain.  These 

2 

algorithms  were  later  refined  and  extended23  to  calculate  C  and  outer  scale 
profiles.  The  computational  flow  detailing  calculation  of  these  profiles  is  as 
follows . 
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From  latitude,  longitude,  and  Greenwich  mean  time  (GMT),  solar  zenith  angle 
may  be  calculated.  From  solar  zenith  angle,  the  amount  of  solar  irradiance  on 
the  earth's  surface  in  the  absence  of  cloud  cover  may  be  estimated  for  the  spec¬ 
ified  location  and  time.24  Knowledge  of  percent  cloud  cover  allows  determina¬ 
tion  of  global  irradiance  and  radiation  class25  and  a  first  estimate  of 
sensible  heat  flux.26  Mean  horizontal  windspeed,  atmospheric  pressure,  and 
temperature,  estimated  for  a  specified  height  within  the  surface  boundary  layer, 
may  be  combined  with  radiation  class  to  estimate  the  Pasquill  fractional  stabil¬ 
ity  parameter.  When  combined  with  terrain  roughness  length,  the  Pasquill 
fractional  stability  parameter  allows  estimation  of  both  the  Obukhov  length  and 
scaling  ratio.27  The  scaling  ratio  appears  as  the  independent  variable  in  the 
dimensionless  wind  shear  and  lapse  rate,  both  basic  quantities  from  the 
similarity  theory.  Two  forms  for  dimensionless  wind  shear  and  lapse  rate  are 
included,  one  proposed  by  Businger28  and  the  other  by  Hansen.29  30  From 
either  of  the  two  forms,  the  corresponding  diabatic  influence  functions  are 
calculated.  An  equation  deriving  from  basic  similarity  relationships  involving 
mean  horizontal  windspeed  and  the  diabatic  influence  function  for  momentum 
allows  calculation  of  friction  velocity.31 

Knowledge  of  a  value  for  temperature,  used  in  the  similarity  equation 
defining  characteristic  temperature,  allows  an  independent  second  estimate  of 
sensible  heat  flux.  To  maintain  consistency  among  the  similarity  parameters,  an 
iterative  scheme  is  employed  to  adjust  friction  velocity  and  Obukhov  length  so 
that  values  for  sensible  heat  flux  predicted  by  similarity  theory  and  solar 
irradiance  are  in  agreement.  From  this  adjusted  set  of  parameters,  profiles  are 
obtained  for  both  potential  temperature  and  potential  temperature  gradient.32 
Sufficient  information  now  exists  to  estimate  profiles  for  the  potential 
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temperature  structure  parameter  and,  ultimately,  the  refractive  index  structure 
parameter . 


Rigorous  development  of  the  theoretical  link  between  the  gradient  profile 
for  potential  temperature  and  the  profile  for  the  refractive  index  structure 
parameter  may  be  found  in  Tatarski33  and  Ishimaru.34  The  refractive  index  of 
the  atmosphere  is  a  function  of  wavelength,  temperature,  pressure,  and  specific 
humidity.  For  optical  and  infrared  wavelengths,  humidity  over  land  is 
negligible,  and  the  refractive  index  may  be  expressed  as35 

1  +  7.52  x  10'V2  ]lO'S  .  (1) 

where  P  is  pressure  in  millibars,  K  is  temperature  in  K  and  A  is  wavelength  in 
micrometers.  This  expression,  when  combined  with  the  mean  potential  temperature 
gradient,  results  in  the  following  approximation  for  the  gradient  of  the  mean 
refractive  index36 


n  -  1  + 


77. 6P 
K 


2n  _  (-77.6  x  10~6p) (1  ±  7,52  x  lO'V2)...  ... 

dz  ~  ,  2  L*  Vn 

kz(9  -  qz) 


(2) 


where  n  is  mean  refractive  index,  9  is  mean  potential  temperature,  z  is  height 
above  ground  in  meters,  a  is  the  adiabatic  lapse  rate,  k  is  von  Karman’s 
constant,  T+  is  characteristic  temperature,  and  is  dimensionless  lapse 
rate.  Used  with  results  from  the  Kolmogorov  principle  of  universal  equilibrium, 
eq.  (2)  allows  calculation  of  profiles  for  the  refractive  index  structure 
parameter. 37 
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3.0  PROPAGATION  STATISTICS  DESCRIBING  SYSTEM  PERFORMANCE 


For  each  propagation  path  through  the  atmosphere,  a  choice  of  statistics  may 
be  estimated  describing  loss  in  system  performance  due  to  optical  turbulence. 
The  statistics  chosen  for  IMTURB  derive  from  a  comprehensive  and  flexible  propa¬ 
gation  model  based  on  weak  fluctuation  theory38  wherein  all  propagation 
statistics  are  weighted  integrals  of  the  refractive  index  structure  parameter 
over  the  propagation  path.  The  four  basic  propagation  statistics  are  (1) 
receiver  coherence  diameter;  (2)  log- amplitude  variance;  (3)  scintillation 
averaging  length;  and  (4)  isoplanatism  effective  path  length.  Subsidiary 
statistics  include  limiting  resolution  source  length,  isoplanatism  source  size, 
and  normalized  intensity  variance.  Other  options  available  to  the  IMTURB  user 
include  (1)  choice  of  plane  or  spherical  wave;  (2)  spatial  power  spectrum  for 
log-amplitude  and  phase  variations;  (3)  fast  and  slow  modulation  transfer  func¬ 
tions;  (4)  imaging  optics  resolution/transmitter  beam  spread;  (5)  heterodyne 
receiver  analysis;  (6)  predetection  compensation  analysis;  and  (6)  normalized, 
aperture -averaged  intensity  receiver  analysis.  The  four  basic  propagation 
statistics  are  described  below. 

The  receiver  coherence  diameter  is  the  most  widely  known  of  the  basic  propa¬ 
gation  statistics.  It  is  a  fundamental  measure  of  wave  front  distortion  and  is 
required  for  estimation  of  such  fundamental  forms  as  the  mutual  coherence  func¬ 
tion  and  modulation  transfer  function.  In  an  atmosphere  free  of  optical  turbu¬ 
lence,  an  imaging  system  has  the  potential  for  virtually  unlimited  increases  in 
resolution  with  corresponding  increases  in  optics  diameter.  However,  when  the 
same  system  is  operated  in  the  presence  of  optical  turbulence,  there  is  a 
diameter  beyond  which  further  size  increases  produce  no  significant  increase  in 
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resolution.  This  diameter  is  reflected  in  the  receiver  coherence  diameter.39 
Reciprocally,  the  degree  of  optical  turbulence- induced  beam  spread  for  focused 
or  collimated  laser  transmitters  may  be  approximated  from  the  receiver  coherence 
diameter  for  spherical  or  plane  wave  propagation,  respectively. 

The  variance  of  the  log- amplitude  ratio  is  a  point  statistic  relating  to 
optical  turbulence- induced  random  fluctuations  in  the  amplitude  of  an  electro¬ 
magnetic  wave.  The  log- amplitude  ratio  is  the  real  part  of  a  complex  stochastic 
integral  describing  electromagnetic  propagation  through  optical  turbulence. 
Derivation  and  interpretation  of  this  integral  are  major  concerns  of  weak 
perturbation  theory.  This  propagation  statistic,  commonly  called  the  log- 
amplitude  variance,40  is  useful  in  predicting  the  onset  of  saturation. 
Saturation  occurs  when  multiple  scattering  among  optical  turbules  becomes 
significant;  weak  fluctuation  theory  is  then  said  to  "saturate." 

The  last  two  basic  propagation  statistics  are  scintillation  averaging  length 
and  isoplanatism  effective  path  length.  The  scintillation  averaging  length41 
describes  the  phenomenon  of  aperture  averaging.  As  "photon  bucket"  type  photo - 
receivers  increase  in  area,  there  is  a  corresponding  decrease  in  the  fluctu¬ 
ations  of  measured  intensity.  The  scintillation  is  said  to  be  "averaged"  over 
the  area.  Apertures  with  diameters  one -tenth  the  scintillation  averaging  length 
exhibit  variations  in  intensity  similar  to  those  predicted  for  a  point,  while 
apertures  on  the  order  of  ten  times  the  scintillation  averaging  length  exhibit 
little  variation  in  intensity.  The  isoplanatism  effective  path  length,  used 
with  receiver  coherence  diameter,  defines  the  isoplanatic  angle  and  hence  the 
isoplanatic  patch.42  Within  a  region  the  size  of  the  isoplanatic  patch,  wave 
front  distortion  is  nearly  constant,  and  image  quality  may  be  improved  by  the 
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removal  of  wave  front  tilt,  Isoplanatic  patch  size  is  useful  in  the  design  of 
adaptive  devices  for  partial  correction  of  optical  turbulence  effects. 

In  addition  to  the  interpretive  value  of  basic  and  subsidiary  propagation 
statistics,  IMTURB  provides  path  specific  fast  and  slow  modulation  transfer 
functions.  These  may  be  used  in  conjunction  with  an  image  processing  system  to 
allow  simulation  of  specified  EO  system  behavior  in  a  controlled,  sterile 
environment . 

4.0  SAMPLE  MODEL  RESULTS 

Two  sample  scenarios  are  described  here  to  illustrate  typical  model  results. 
In  the  first  scenario,  Case  A,  an  EO  imaging  sensor  operating  at  1.06  ^m  and 
located  2  m  above  the  ground  observes  a  target  500  m  distant  and  11  m  above  the 
ground.  Spherical  wave  propagation  is  appropriate  with  the  source  at  the 
target.  The  site  is  White  Sands  Missile  Range  (WSMR) ,  New  Mexico  (USA),  located 
at  latitude  32°  north,  longitude  106°  west.  The  date  is  31  May  1984;  the  time 
is  1300  mountain  standard  time  (MST) ,  translating  to  a  GMT  of  2000.  The  day  is 
cloudless;  the  upwind  terrain  is  a  broad  region  of  desert  brush  and  scrub, 
having  a  tabulated  roughness  length  of  16  cm.  From  a  nearby  observation  tower 
with  sensors  located  10  m  above  the  ground,  air  temperature  is  35  “C;  pressure 
is  850  mbar  (the  altitude  of  the  Tularosa  Basin  where  the  WSMR  is  located  is 
4000  ft  above  sea  level);  and  the  average  windspeed  is  3  m/s.  IMTURB 
characterizes  the  atmosphere  as  strongly  unstable,  with  the  height  of  the 
surface  boundary  layer  estimated  at  11  m.  The  set  of  basic  and  subsidiary 
propagation  statistics  for  Case  A  is  given  in  table  1.  Note  that  log- amplitude 
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variance  is  0.206,  indicating  that  Case  A  is  within  the  limits  of  weak 
fluctuation  theory. 


Table  1.  Basic  and  Subsidiary  Propagation  Statistics  for  Case  A 


Basic  Propagation  Statistics 

*  Receiver  Coherence  Diameter 

1.54 

centimeters 

*  Log-Amplitude  Variance 

0.206 

nepers  -  squared 

*  Scintillation  Averaging  Length 

2.78 

centimeters 

*  Isoplanatism  Effective  Path  Length 

117.7 

meters 

Subsidiary  Propagation 

Statistics 

*  Limiting  Resolution  Source  Length 

3.89 

centimeters 

*  Isoplanatism  Source  Size 

3.26 

centimeters 

*  Normalized  Intensity  Variance 

1.28 

The  conditions  chosen  for  Case  B  illustrate  the  significant  difference  in 
optical  turbulence  structure  arising  from  relatively  minor  changes  in  environ¬ 
mental  conditions.  The  location,  terrain,  time,  temperature,  and  pressure  are 
identical  to  Case  A.  However,  cloud  cover  has  increased  to  25  percent  and 
windspeed  to  5  m/s.  IMTURB  predicts  a  moderately  unstable  atmosphere  with  the 
height  of  the  surface  boundary  layer  near  43  m.  The  basic  and  subsidiary 
propagation  statistics  for  Case  B  are  listed  in  table  2,  from  which  we  see  that 
log-amplitude  variance  is  reduced  to  0.092,  well  within  the  bounds  of  weak  fluc¬ 
tuation  theory. 
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Table  2.  Basic  and  Subsidiary  Propagation  Statistics  for  Case  B. 


Basic  Propagation  Statistics 

* 

Receiver  Coherence  Diameter 

2.65 

centimeters 

* 

Log-Amplitude  Variance 

0.092 

nepers  -  squared 

* 

Scintillation  Averaging  Length 

2.93 

centimeters 

* 

Isoplanatism  Effective  Path  Length 

126.0 

meters 

Subsidiary  Propagation 

Statistics 

* 

Limiting  Resolution  Source  Length 

2.26 

centimeters 

* 

Isoplanatism  Source  Size 

5.25 

centimeters 

* 

Normailized  Intensity  Variance 

0.44 

Due  to  space  limitations,  the  variety  of  atmospheric  stability  and  similar¬ 
ity  parameters  descriptive  of  the  micrometeorology  have  been  omitted.  However, 
we  have  included  profiles  for  the  turbulent  inner  and  outer  scales  and  for  the 
refractive  index  structure  parameter.  These  profiles  characterize  the  optical 
turbulence  structure  over  the  region  encompassing  the  propagation  path  and  are 
adequate  even  for  advanced  applications.  The  three  profiles  for  Cases  A  and  B 
are  listed  in  figs.  1,  2,  and  3.  All  profiles  are  terminated  at  the  maximum 
extent  of  the  surface  boundary  layer,  by  definition  the  absolute  value  of  the 
Obukhov  length.  Note  that  outer  scale  follows  the  rule  of  thumb  for  unstable 
atmospheres,  that  is,  outer  scale  approximately  equals  height  above  ground. 
Inner  scale  exhibits  the  dependence  on  wind  velocity  observed  by  numerous 

investigators.43  The  severity  of  optical  turbulence  can  be  determined  from  the 
2 

C^  profiles,  as  shown  in  fig.  3.  Notice  that  optical  turbulence  strength 
decreases  rapidly  with  height  in  both  cases.  For  these  examples,  one  would 
expect  significant  degradation  in  performance  for  EO  systems  operating  in  the 
visible  to  near  infrared  (IR),  and  in  the  far  IR  over  longer  ranges.  Values  for 
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Height  (meters) 


the  receiver  coherence  diameter  of  1.5  cm  and  2.6  cm  for  Cases  A  and  B,  respect 
ively,  attest  to  this  fact. 


0  2  4  6  8 

Inner  Scale  (millimeters) 


Figure  1.  Plot  of  the  Turbulent  Inner  Scale  with  Height  for  Cases  A  and  B 


Related  to  the  receiver  coherence  diameter  is  the  subsidiary  statistic 
limiting  resolution  source  size.  This  is  the  maximum  resolution  possible  for 
the  propagation  path  and  wavelength  regardless  of  optics  size.  In  terms  of 
laser  performance,  the  limiting  resolution  source  size  is  the  "spot"  size  of  a 
laser  focused  to  a  point.  In  Case  A,  the  value  is  3.9  cm;  for  Case  B,  2.3  cm. 
In  the  former  case,  a  sensor  would  be  able  to  resolve  nothing  smaller  than  3.9 
cm  on  the  target  at  500  m.  In  Case  B  the  same  system  can  resolve  details  as 
small  as  2.3  cm. 

Figure  4  provides  a  comparative  plot  of  log- amplitude  variance  with  range 
for  Cases  A  and  B.  The  increase  in  range  of  validity  for  weak  fluctuation 
theory  in  the  latter  case  is  noticeable.  The  relatively  short  distances  to 
saturation  in  comparison  with  normal  engagement  ranges  are  a  strong  incentive  to 
extend  model  results  by  use  of  strong  perturbation  theory.  It  is  also 
instructive  to  plot  receiver  coherence  diameter  versus  range  for  Cases  A  and  B. 
Referring  to  fig.  5,  we  see  that  as  range  increases,  the  integrated  effect  of 
optical  turbulence  is  evident,  as  is  the  consistently  superior  performance  of 
the  system  operated  under  the  environmental  conditions  of  Case  B. 
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Figure  4.  Plot  of  the  Log- Amplitude  Variance  with  Range,  Extended  to  Satura¬ 
tion.  The  Sensor  is  at  2  m  and  Target  is  at  11  m  (Cases  A  and  B) 


Range  (meters) 

Figure  5.  Plot  of  the  Receiver  Coherence  Diameter  with  Range  for  a  Sensor  at 
2m  and  Target  at  11  m  (Cases  A  and  B) 
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An  interesting  property  of  certain  propagation  statistics  that  occur  as 
integrals  over  the  propagation  path  is  the  phenomenon  of  path  weighting.  For 
spherical  wave  propagation,  receiver  coherence  diameter  exhibits  this  trait. 
Consider  a  scenario  identical  to  Case  A  except  that  the  sensor  and  target  posi¬ 
tions  are  reversed.  The  receiver  coherence  diameter  calculated  for  a  sensor  at 
11  m  and  target  at  2  m,  with  a  path  length  of  500  m,  is  2.6  cm.  This 
represents,  coincidentally,  the  increase  in  system  performance  of  Case  B  over 
Case  A.  A  plot  of  receiver  coherence  diameter  versus  range  for  the  two 
sensor/target  geometries  is  given  in  fig.  6. 
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Figure  6.  Plot  of  Receiver  Coherence  Diameter  versus  Range  for  Case  A,  and  with 
Target  and  Optics  Positions  Reversed 
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Perhaps  Che  most  exciting  application  of  the  IMTURB  model  arises  from  its 
use  in  simulating  degradation  due  to  optical  turbulence  using  digital  image 
processing  techniques.44  45  The  U.S.  Army  Atmospheric  Sciences  Laboratory 
(ASL)  has  incorporated  an  IMTURB  clone  into  ASL's  library  of  models  for  simula¬ 
tion  of  atmospheric  effects  on  system  performance,  called  BATL.  BATL  is  cur¬ 
rently  under  development  at  the  ASL  Image  Processing  Laboratory  and  employs  a 
Gould  IP8500  with  real  time  digital  disk  and  VAX  11/780  host  computer.  Examples 
of  image  degradation  due  to  optical  turbulence  performed  in  near  real-time  using 
a  convolution  technique  are  given  below.  Figure  7  shows  an  image  of  a  tank 
taken  at  a  far-IR  wavelength  over  a  2330-m  path.  Figure  8  is  the  same  image, 
viewed  over  a  horizontal  path  through  moderate  optical  turbulence.  If  the 
sensor  is  raised  about  10  ra  above  the  ground,  we  see  that  optical  turbulence 
effects  are  reduced,  as  shown  in  fig.  9.  Figure  10  demonstrates  the  blurring 
effect  severe  optical  turbulence  produces. 


Figure  7.  Image  of  a  Tank  Taken  at  a  Far-IR  Wavelength  Over  a  2330-m  Path. 
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Figure  8. 


Same  Image  as  Above  but  Viewed  Over  a  Horizontal  Path  Through 
Moderate  Optical  Turbulence 


Figure  9. 


Optical  Turbulence  Effects  are  Reduced  When  the  Sensor  is  Raised 
About  10  m  Above  the  Ground 


210 


literature  cited 


Andrews,  L.  C.,  and  R.  L.  Phillips,  1989:  "New  temperature  spectrum  model  based 
on  fluctuations  in  the  average  dissipation  rated,"  Propagation  Engineering. 
N.  S.  Kopeika  and  W.  B.  Miller,  eds,  Proceedings  SPIE  1115. 

Batchelor,  G.  K. ,  1947:  "Kolmogorof f ' s  theory  of  locally  isotropic  turbulence," 
Proceedings  Camb  Phil  Soc.  43,  533-559. 

Busch,  N.  E.  ,  1973:  "On  the  mechanics  of  atmospheric  turbulence,"  Workshop_Q_n 
Micrometeorologv .  D.  A.  Haugen,  ed,  pp .  1-65,  American  Meteorological 
Society. 

Businger,  J.  A.,  1973:  "Turbulent  transfer  in  the  atmospheric  surface  layer," 
Workshop  on  Micrometeorologv.  D.  A.  Haugen,  ed,  pp .  67-100,  American 
Meteorological  Society. 

Clifford,  S.  F. ,  1978:  "The  classical  theory  of  wave  propagation  in  a  turbulent 
medium,"  Laser  Beam  Propagation  in  a  Turbulent  Atmosphere.  J.  W.  Strohbehn, 
ed,  pp  9-44,  Springer-Verlag,  New  York. 

Durack,  D.  L.,  and  M.  K.  Giles,  1988:  "Method  to  compute  space-variant  image 
degradation  by  atmospheric  turbulence  ,  "  Optical.  Infrared  and  Millimeter 
Wave  Propagation  Engineering.  W.  B.  Miller  and  N.  S.  Kopeika,  eds, 
Proceedings  SPIE  926,  286-295. 

Fried,  D.  L. ,  1966:  "Optical  resolution  through  a  randomly  inhomogeneous  medium 
for  very  long  and  very  short  exposures,"  J  Opt  Soc  Am.  56,  1372. 

Fried,  D.  L.  ,  1967:  "Aperature  averaging  of  scintillation,"  J  Opt  Soc  Am.  57, 
169. 

Fried,  D.  L. ,  1974:  Theoretical  Study  of  Non-Standard  Imaging  Concepts.  Interim 
Technical  Report  on  RADC  Contract  No  F30602-74-C-0115 ,  RADC  Report  No  TR-74- 
185  and  RADC  Report  No  TR-74-276,  Rome  Air  Development  Center,  Griffiss  Air 
Force  Base,  NY. 

Fried,  D.  L.  ,  1974:  Theoretical  Study  of  Non-Standard  Imaging  Concepts.  RADC- 
TR-76-51  (AD  A023627) ,  Rome  Air  Development  Center,  Griffiss  Air  Force  Base, 
NY. 

Fried,  D.  L.  ,  Theoretical  Analysis  of  Aperature  Averaging.  Optical  Science 
Consultants  Report  No.  DR-015,  Final  Report  of  NASA  Contract  NAS5-23272. 

Giles,  M.  K.  ,  and  D.  L.  Durack,  1988:  "A  convolution  model  of  long-exposure 
image  blur  due  to  atmospheric  turbulence,"  Optical  Infrared  and  Millimeter 
Wave  Propagation  Engineering.  W.  B.  Miller  and  N.  S.  Kopeika,  eds, 
Proceedings  SPIE  926,  296-304. 

Goodman,  J.  W.  ,  1985:  Statistical  Optics,  pp  361-457,  John  Wiley  &  Sons,  New 
York. 

Hansen,  F.  V.  ,  1977:  The  Critical  Richardson  Number.  ECOM-5829,  U.S.  Army 
Atmospheric  Sciences  Laboratory,  White  Sands  Missile  Range,  NM. 


212 


Hansen,  F.  V.  ,  1980:  Flux  Profile  Relationships  for  Development  of  Standards  of 
Comparison .  Internal  Report,  U.S.  Army  Atmospheric  Sciences  Laboratory, 
White  Sands  Missile  Range,  NM. 

Hill,  R.  J.,  1978:  "Models  of  the  scalar  spectrum  for  turbulent  advection,"  J 
Fluid  Mech  88(3),  541-562. 

Ishimaru,  A.  ,  1978:  Wave  Propagation  and  Scattering  in  Random  Media.  Vol  I  and 
II,  Academic  Press,  New  York. 

Kolmogoroff,  A.  N.,  1941:  "Dissipation  of  energy  in  the  locally  isotropic 
turbulence,"  C  R  Acad  Sci .  U.S.S.R.,  32(1),  16-18. 

Kolmogoroff,  A.  N. ,  1941:  "The  local  structure  of  turbulence  in  incompressible 
viscous  fluid  for  very  large  Reynolds’  numbers,"  C  R  Acad  Sci .  U.S.S.R., 
30(4),  301-305. 

Rung,  E.  C. ,  1963:  "Climatology  of  aerodynamic  roughness  parameter  and  energy 

dissipation  in  the  planetary  boundary  layer  of  the  northern  hemisphere," 
Annual  Report,  Studies  of  the  Effects  of  Variations  in  Boundary  Conditions 
on  the  Atmospheric  Boundary  Laver.  Contract  DA- 36 -0  39  -  AMC  -  008  78  -  0  , 
Department  of  Meteorology,  University  of  Wisconsin,  Madison,  WI . 

Liu,  M.,  D.  Durran,  P.  Mundkur,  M.  Yocke ,  J.  Ames,  1976:  The  Chemistry, 
Dispersion  and  Transport  of  Air  Pollutants  Emitted  From  Fossil  Fuel  Power 

Plants  in  California: _ Data  Analysis  and  Emission  Impact  Model.  Contract 

Report  No.  EF76-18R,  Systems  Applications,  Incorporated,  San  Rafael,  CA. 

Miller,  W.  B.,  J.  C.  Ricklin,  and  W.  J.  Stewart,  1987:  An  Optical  Turbulence 
Code  for  the  Surface  Boundary  Laver"  ASL-TR-0220,  U.S.  Army  Atmospheric 
Sciences  Laboratory,  White  Sands  Missile  Range,  NM. 

Miller,  W.  B. ,  and  J.  C.  Ricklin,  1988:  "Description  of  optical  turbulence 
effects  on  propagation  in  the  atmospheric  surface  boundary  layer,"  Optical . 
Infrared  and  Millimeter  Wave  Propagation  Engineering.  W.  B.  Miller  and  N.  S. 
Kopeika,  eds ,  Proceedings  SPIE  926,  268-277. 

Miller,  W.  B.  ,  and  J.  C.  Ricklin,  1989:  IMTURB:  A  Module  for  Imaging  Through 
Optical  Turbulence.  ASL - TR -  0 2 2  1  -  2  7  ,  U.S.  Army  Atmospheric  Sciences 
Laboratory,  White  Sands  Missile  Range,  NM.  (in  preparation) 

Obukhoff ,  A.  ,  1941:  "On  the  energy  distribution  in  the  spectrum  of  a  turbulent 
flow,"  C  R  Acad  Sci.  U.S.S.R.,  33(1),  19-21. 

Ochs,  G.  R.  ,  1989:  formerly  with  N0AA/WPL,  Privately  communicated  through  D. 
Tofsted,  U.S.  Army  Atmospheric  Sciences  Laboratory,  White  Sands  Missile 
Range ,  NM . 

Ochs,  G.  R.  ,  and  R.  J.  Hill,  1985:  "Optical-scintillation  method  of  measuring 
turbulence  inner  scale,"  AppI  Opt.  24(15),  2430-2332. 


Note:  Articles  were  translated  using  the  spelling  "Kolmogoroff,"  while  modern 
usage  has  name  spelled  "Kolmogorov."  Thus,  we  use  "Kolmogorov"  in  the  text,  and 
"Kolmogoroff"  in  references. 


213 


Paulson,  C.  A.,  1970:  "The  mathematical  representation  of  wind  speed  and 
temperature  profiles  in  an  unstable  atmospheric  surface  layer,"  J  Appl 
Meteorol .  9,  857-861. 

Ricklin,  J.  C.,  1986:  The  Impact  of  Optical  Turbulence  on  Target  Acquisition. 
ASL-TR-0209,  U.S.  Army  Atmospheric  Sciences  Laboratory,  White  Sands  Missile 
Range ,  NM . 

Smith,  F.  B.  ,  1972:  "Proceedings  of  the  Third  Meeting  of  the  Expert  Panel  on 
Air  Pollution  Modeling,"  NATO  Committee  on  the  Challenges  of  Modern  Society, 
ch  XVII,  Paris,  France. 

Smith,  F.  B.,  1979:  "The  relation  between  Pasquill  stability  P  and  Kazanski- 
Monin  stability  (in  neutral  and  unstable  conditions),"  Atmos  Environ . 
Pergamon  Press  Ltd.,  United  Kingdom  13,  879-881. 

Smith,  F.  B.  ,  and  R.  M.  Blackhall,  1979:  "The  application  of  field  experiment 
data  to  the  parameterization  of  plumes  from  ground  level  and  elevated 
sources,"  Mathematical  Modeling  of  Turbulent  Diffusion  in  the  Environment. 
C.  J.  Harris,  ed,  Academic  Press,  New  York. 


Stewart,  W.  J.,  1985:  An  HP-9825  Computer  Model  for  Turbulent  Velocity 
Spectra .  ASL-CR-85-0100-2 ,  U.S.  Army  Atmospheric  Sciences  Laboratory,  White 
Sands  Missile  Range,  NM. 

Tatarski,  V.  I.,  1961:  Wave  Propagation  in  a  Turbulent  Medium,  trans ,  R.  A. 
Silverman,  McGraw-Hill  Book  Co.,  New  York. 

Wyngaard,  J.  C.,  1973:  "On  surface  layer  turbulence,"  Workshop  on 

Micrometeorology .  ed,  D.  A.  Haugen,  pp .  101-149,  American  Meteorological 
Society. 


214 


ENDNOTES 


1  A.  N.  Kolmogoroff,  "The  local  structure  of  turbulence  in  incompressible 
viscous  fluid  for  very  large  Reynolds'  numbers,"  C .  R.  Acad.  Sci .  .  U.S.S.R. 
30(4) ,  301-305  (1941) . 

2  A.  N.  Kolmogoroff,  "Dissipation  of  energy  in  the  locally  isotropic 
turbulence,"  C,  R.  Acad.  Sci..  U.S.S.R.  32(1),  16-18  (1941). 

3  G.  K.  Batchelor,  "Kolmogoroff ' s  theory  of  locally  isotropic  turbulence," 
Proc .  Camb .  Phil .  Soc . .  43,  533-559  (1947). 

4  V.  I.  Tatarski,  Wave  Propagation  in  a  Turbulent  Medium,  trans  .  R.  A. 
Silverman,  McGraw-Hill  Book  Co.,  New  York  (1961). 

5  W.  B.  Miller  and  J.  C.  Ricklin,  U.S.  Army  Atmospheric  Sciences  Laboratory, 
White  Sands  Missile  Range,  New  Mexico,  IMTURB:  A  Module  for  Imaging  Through 
Optical  Turbulence  (1989),  ASL-TR-0221-27 .  (in  preparation) 

6  J.  C.  Ricklin,  U.S.  Army  Atmospheric  Sciences  Laboratory,  White  Sands 
Missile  Range,  New  Mexico,  The  Impact  of  Optical  Turbulence  on  Target 
Acquisition  (1986),  ASL-TR-0209. 

7  Tatarski,  V.  I.,  op.  cit. 

8  A.  Ishimaru,  Waye  Propagation  and  Scattering  in  Random  Media.  Vol.  I  and  II, 
Academic  Press,  New  .ork  (1978). 

9  S.  F.  Clifford,  "The  classical  theory  of  wave  propagation  in  a  turbulent 
medium,"  Laser  Beam  Propagation  in  a  Turbulent  Atmosphere.  J.  W.  Strohbehn,  ed, 
pp.  9-44,  Springer-Verlag,  New  York  (1978). 

10  J.  W.  Goodman,  Statistical  Optics,  pp.  361-457,  John  Wiley  &  Sons,  New  York 
(1985V 

11  A.  Obukhoff,  "On  the  energy  distribution  in  the  spectrum  of  a  turbulent 
flow,"  C.  R.  Acad.  Sci..  U.S.S.R.,  33(1),  19-21  (1941). 

12  N.  E.  Busch,  "On  the  mechanics  of  atmospheric  turbulence,"  in  Workshop  on 
Micro  meteorology.  D.  A.  Haugen,  ed. ,  pp .  1-65,  American  Meteorological  Society 
(1973). 

13  J.  A.  Businger,  "Turbulent  transfer  in  the  atmospheric  surface  layer,"  in 

Workshop  on  Micrometeorologv  .  D.  A.  Haugen,  ed.  ,  pp  .  67  -  100,  American 

Meteorological  Society  (1973). 

14  J.  C.  Wyngaard,  "On  surface  layer  turbulence,"  in  Workshop  on 
Micrometeorology .  D.  A.  Haugen,  ed. ,  pp.  101-149,  American  Meteorological 
Society  (1973). 

15  Ishimaru,  A.,  op.  cit.,  Vol.  II,  Appendix  C. 

18  G.  R.  Ochs  and  R.  J.  Hill,  "Optical- scintillation  method  of  measuring 
turbulence  inner  scale,"  AppI.  Opt. .  24(15),  2430-2332  (1985). 


215 


17  Ishimaru,  A.,  op.  cit. ,  pp.  420-422. 

18  R.  J.  Hill,  "Models  of  the  scalar  spectrum  for  turbulent  advection,"  J , 
Fluid  Mech. .  88(3),  541-562  (1978). 

19  L.  C.  Andrews  and  R.  L.  Phillips,  "New  temperature  spectrum  model  based  on 
fluctuations  in  the  average  dissipation  rates,"  in  Propagation  Engineering.  N. 
S.  Kopeika  and  W.  B.  Miller,  eds.,  Proc.  SPIE  1115  (1989). 

20  Miller,  W.  B. ,  and  J.  C.  Ricklin,  op.  cit.,  Appendix  B. 

21  E.  C.  Rung,  Dept,  of  Meteorology,  University  of  Wisconsin,  Madison,  WI , 
"Climatology  of  aerodynamic  roughness  parameter  and  energy  dissipation  in  the 
planetary  boundary  layer  of  the  northern  hemisphere,"  in  Annual  Report  Studies 
of  the. Effects  of  Variations  in  Boundary  Conditions  on  the  Atmospheric  Boundary 
Laver  (1963),  Contract  DA-36-039-AMC-00878-0. 

22  W.  J.  Stewart,  U.S.  Army  Atmospheric  Sciences  Laboratory,  White  Sands 
Missile  Range,  New  Mexico,  An  HP-9825  Computer  Model  for  Turbulent  Velocity 
Spectra  (1985),  ASL-CR-85-0100-2 . 

23  W.  B.  Miller,  J.  C.  Ricklin,  and  W.  J.  Stewart,  U.S.  Army  Atmospheric 
Sciences  Laboratory,  White  Sands  Missile  Range,  New  Mexico,  An  Optical 
Turbulence  Code  for  the  Surface  Boundary  Laver  (1987),  ASL-TR-0220. 

24  Miller,  W.  B. ,  and  J.  C.  Ricklin,  op.  cit.,  Appendix  C. 

25  F.  B.  Smith,  "The  relation  between  Pasquill  stability  P  and  Kazanski -Monin 
stability  (in  neutral  and  unstable  conditions),"  in  Atmospheric  Environment. 
Pergamon  Press  Ltd.,  United  Kingdom  13,  879-881  (1979). 

29  F.  B.  Smith,  "Proceedings  of  the  Third  Meeting  of  the  Expert  Panel  on  Air 
Pollution  Modeling,"  in  NATO  Committee  on  the  Challenges  of  Modern  Society 
(Paris,  France,  1972),  ch.  XVII. 

27  M.  Liu  et  al.,  National  Technical  Information  Center,  The  Chemistry, 
Dispersion  and  Transport  of  Air  Pollutants  Emitted  From  Fossil  Fuel  Power  Plants 
in  California:  Data  Analysis  and  Emission  Impact  Model.  Contract  Report  No. 
EF76-18R,  (1976). 

28  Businger,  J.  A.,  op.  cit. 

29  F.  V.  Hansen,  U.S.  Army  Atmospheric  Sciences  Laboratory,  White  Sands  Missile 
Range,  New  Mexico,  Flux  Profile  Relationships  for  Development  of  Standards  of 
Comparison  (1980),  Internal  Report. 

30  F.  V.  Hansen,  U.S.  Army  Atmospheric  Sciences  Laboratory,  White  Sands  Missile 
Range,  New  Mexico,  The  Critical  Richardson  Number  (1977),  ECOM-5829. 

31  C.  A.  Paulson,  "The  mathematical  representation  of  wind  speed  and 
temperature  profiles  in  an  unstable  atmospheric  surface  layer,"  J  ,  AppI 
Meteorol. .  9,  857-861  (1970). 


216 


32  F.  B.  Smith  and  R.  M.  Blackhall,  "The  application  of  field  experimant  data 
to  the  parameterization  of  plumes  from  ground  level  and  elevated  sources,"  in 
Mathematical  Modeling  of  Turbulent  Diffusion  in  the  Environment .  C.  J.  Harris, 
ed. ,  Academic  Press,  New  York  (1979). 

33  Tatarski,  V.  I.,  op.  cit. 

34  Ishimaru,  A.,  op.  cit. 

35  Clifford,  S.  F. ,  op.  cit. 

38  W.  B.  Miller  and  J.  C.  Ricklin,  "Description  of  optical  turbulence  effects 
on  propagation  in  the  atmospheric  surface  boundary  layer,"  in  Optical.  Infrared 
and  Millimeter  Wave  Propagation  Engineering.  W.  B.  Miller  and  N.  S.  Kopeika, 
eds.,  Proc.  SPIE  926,  268-277  (1988). 

37  Miller,  W.  B. ,  and  J.  C.  Ricklin,  op.  cit. 

38  D.  L.  Fried,  Rome  Air  Development  Center,  Griff iss  Air  Force  Base,  New  York, 
Theoretical  Study  of  Non-Standard  Imaging  Concepts  (1976),  RADC-TR-76-51  (AD 
A023627) . 

39  D.  L.  Fried,  "Optical  resolution  through  a  randomly  inhomogeneous  medium  for 
very  long  and  very  short  exposures,"  J .  Pot .  Soc .  Am. .  56,  1372  (1966). 

40  D.  L.  Fried,  "Aperature  averaging  of  scintillation,"  J .  Opt .  Soc .  Am. .  57, 
169  (1967). 

41  D.  L.  Fried,  Theoretical  Analysis  of  Aperture  Averaging.  Optical  Science 
Consultants  Report  No.  DR-015,  Final  Report  on  NASA  Contract  NAS5- 23272. 

42  D.  L.  Fried,  Rome  Air  Development  Center,  Griff iss  Air  Force  Base,  New  York, 
Theoretical  Study  of  Non-Standard  Imaging  Concepts.  Interim  Tech.  Report  on  RADC 
Contract  No.  F30602-74-C-0115 ,  RADC  Report  No.  TR-74-185  (1974)  and  RADC  Report 
No.  TR-74-276  (1974). 

43  G.  R.  Ochs,  formerly  with  NOAA/WPL,  Privately  communicated  through  D. 
Tofsted,  U.S.  Army  Atmospheric  Sciences  Laboratory,  White  Sands  Missile  Range. 
New  Mexico,  (1989). 

44  D.  L.  Durack  and  M.  K.  Giles,  "Method  to  compute  space -variant  image 
degradation  by  atmospheric  turbulence,"  in  Optical.  Infrared  and  Millimeter  Wave 
Propagation  Engineering.  W.  B.  Miller  and  N.  S.  Kopeika,  eds.,  Proc.  SPIE  926, 
286-295  (1988)  . 

45  M.  K.  Giles  and  D.  L.  Durack,  "A  convolution  model  of  long-exposure  image 
blur  due  to  atmospheric  turbulence,"  in  Optical.  Infrared  and  Millimeter  Wave 
Propagation  Engineering.  W.  B.  Miller  and  N.  S.  Kopeika,  eds.,  Proc.  SPIE  926, 
296-304  (1988)  . 


217 


THIS  PAGE  INTENTIONALLY  BLANK 


218 


Presented  at  the 

DIGITAL /ELECTRONIC  TERRAIN  BOARD  SYMPOSIUM 
5-6  October  1989  GACIAC  PR  89-07 


ADAPTIVE  TERRAIN  MODELS 
FOR  REAL-TIME  SIMULATION 


Lori  L.  Scarlatos 

Grumman  Data  Systems,  M.S.  D12-237 
1000  Woodbury  Road 
Woodbury,  NY  11797 


ABSTRACT 

Flexibility  and  speed  of  the  digital/electronic  terrain  board  simulations  depends  on  the 
cartographic  data  structures  used  to  produce  simulations.  We  have  consequently  identified  three 
primary  requirements  for  terrain  data  structures  to  support  maximum  flexibility  and  speed. 
Specifically,  data  structures  must  support  (1)  multiple  levels  of  detail,  (2)  fast  spatial  search,  and 
(3)  data  compression  without  loss  of  accuracy.  We  present  two  hierarchical  terrain  data  struc¬ 
tures  which  support  these  capabilities.  The  first  structure  is  a  quadtree  built  via  pyramidal 
decomposition.  The  second  structure  is  a  hierarchical  arrangement  of  triangulated  irregular  net¬ 
works  (TINs).  In  this  paper  we  describe  both  structures,  discussing  the  benefits  and  limitations 
of  both,  and  presenting  implementation  results. 


1.  INTRODUCTION 

Designed  to  simulate  the  real  world  in  three  or  more  dimensions  and  in  ail  useful  spec- 
trums.  the  digital/electronic  terrain  board  requires  cartographic  data  structures  that  will  support 
its  proposed  capabilities.  Cartographic  data  structures  significantly  effect  the  speed  and  flexibil¬ 
ity  with  which  data  may  be  accessed,  processed,  and  displayed.  Therefore,  careful  consideration 
must  be  given  to  these  structures.  A  recent  report  out  of  Rome  Air  Development  Center 
[Tesser88]  identified  three  primary  data  structure  requirements  for  fast  and  flexible  3D  visualiza¬ 
tion  of  cartographic  data:  (1)  multiple  levels  of  detail,  (2)  fast  spatial  search,  and  (3)  data 
compression  without  loss  of  accuracy.  These  requirements  are  equally  applicable  to  the 
digital/electronic  terrain  board. 

Of  all  the  cartographic  entities  used  by  digital  cartographic  applications,  terrain  topol¬ 
ogy  is  the  most  important,  providing  a  foundation  upon  which  all  other  data  sources  are  based. 
Most  digital  simulator  systems  rely  on  terrain  data  stored  in  a  grid  format  such  as  the  Defense 
Mapping  Agency’s  Digital  Terrain  Elevation  Data  (DMA  DTED).  Grid  formats  store  elevation 
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values  in  a  two-dimensional  matrix.  Row  and  column  positions  in  the  matrix  correspond  to  lati¬ 
tude  and  longitude  on  the  earth,  with  a  fixed  increment  between  the  measured  elevation  samples 
(posts).  Perspective  views  of  a  terrain  grid  may  be  generated  from  a  regular  triangular  mesh  pro¬ 
duced  by  connecting  the  posts  with  triangle  edges. 

We  have  chosen  to  focus  on  terrain  data  structures  for  the  digital/electronic  terrain 
board.  In  the  remainder  of  this  section  we  discuss  the  three  data  structure  requirements  listed 
above,  showing  that  the  traditional  grid  structure  is  inadequate.  We  then  present  two  alternative 
data  structures  that  do  meet  the  requirements:  quadtrees  and  refined  triangulation  hierarchies. 
The  second  section  describes  quadtrees:  what  quadtrees  are,  how  we  build  and  store  quadtrees 
representing  terrain,  and  our  implementation  results.  The  third  section  similarly  describes 
refined  triangulation  hierarchies.  The  section  after  that  explores  trade-offs  of  the  two  data  struc¬ 
tures.  and  the  final  section  draws  some  conclusions  about  the  work  that  we  have  done. 

1.1  MULTIPLE  LEVELS  OF  TERRAIN  DETAIL 

Different  points  of  view,  as  well  as  different  situations,  require  different  levels  of  detail 
in  terrain  representaion.  High  resolution  data  provides  essential  detail  to  applications  in  which 
relatively  small  surface  changes  are  important  in  a  localized  area.  Some  examples  are  low-level 
flight  simulation,  radar  simulation,  and  trafficabilitv,  visibiitv.  and  cross  country  movement 
analysis.  On  the  other  hand,  lower  resolutions  enable  faster  processing  and  display  of  data  cov¬ 
ering  a  larger  gaming  area.  Applications  with  a  global  scope  —  such  as  C'l  mission  planning 
and  long  distance  flight  plans  —  therefore  benefit  from  the  use  of  lower  resolution  data. 
Although  low  resolutions  contain  less  information,  they  must  accurately  retain  critical  terrain 
information,  losing  only  insignificant  details.  Otherwise  the  resulting  representation  will  be 
misleading. 

Because  there  are  benefits  and  drawbacks  to  selecting  any  level  of  detail,  mixed  resolu¬ 
tions  will  be  useful  to  the  digital/electronic  terrain  board.  Combining  various  levels  of  detail  can 
increase  the  scope  of  a  view  without  significantly  increasing  the  number  of  polygons.  Close- 
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range  or  tactically  important  areas  may  be  represented  with  high  levels  of  detail,  while  peri¬ 
pheral  areas  need  only  generalized  representation  with  low  levels  of  detail.  Such  fish-eve  views 
provide  fast  visualizations  with  required  levels  of  detail  without  overloading  the  display  buffer. 

Multiple  levels  of  detail  in  gridded  data  are  commonly  achieved  by  modifying  the  dis¬ 
tance  between  posts.  However,  because  they  do  not  adaptively  conform  to  the  terrain,  gridded 
data  are  inadequate  for  applications  in  which  accuracy,  processing  time,  or  a  smooth  transition 
between  resolutions  is  important.  For  example,  small  post  increments  for  high  levels  of  detail 
result  in  a  very  large  number  of  polygons  which  may  or  may  not  be  essential  to  the  terrain 
representation.  This  is  a  problem  because  any  unnecessary  polygons  will  unnecessarily  increase 
processing  time  for  3D  display  or  sensor  simulation.  On  the  other  hand,  low  levels  of  detail 
with  greater  distances  between  posts  ignore  terrain  characteristics.  Critical  features  may  there¬ 
fore  appear  to  move,  change  size  or  shape,  or  even  disappear  altogether  in  the  lower  resolutions. 
And  because  there  is  no  smooth  transition  between  resolutions,  terrain  features  can  fluctuate  or 
even  pop  in  and  out  of  view,  thereby  distracting  the  viewer. 

Figures  1  and  2  illustrate  some  of  these  problems.  We  selected  a  256x256  grid  of 
DTED-1  elevation  posts  —  representing  less  than  15x15  arc  minutes  over  Nevada  -  as  a  test 
region.  Figure  1  shows  a  perspective  view  of  that  area,  represented  by  a  mesh  of  131.072  trian¬ 
gles.  We  then  subsampled  the  data  three  times,  producing  meshes  over  the  same  area  but  with 
approximately  10000.  5000.  and  2500  triangles  respectively.  Figure  2  shows  perspective  render¬ 
ings  of  these  subsampled  grids,  using  the  same  perspective  view  as  figure  1.  Notice  how  the 
square  hill  in  the  foreground  becomes  a  cone,  and  then  shifts  off  of  the  page.  This  anomaly 
becomes  even  more  apparent  in  dynamic  simulations. 

1.2  FAST  SPATIAL  SEARCH 

Fast  data  search  and  retrieval  reduce  the  time  required  to  produce  a  single  simulated 
sensor  image,  and  will  make  dynamic  simulations  possible  in  the  digital/electronic  terrain  board. 
For  example,  we  must  rapidly  access  elevation  data  at  successive  points  along  a  route  to  simu- 
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late  targets  moving  over  the  terrain.  Point  location  is  also  important  for  querying  the  data  base 
about  a  position,  and  for  interactively  inserting  and  removing  targets  in  "what-if"  scenarios.  Fast 
range  retrieval  reduces  time  spent  producing  multiple  resolution  terrain  representations  and  map¬ 
ping  images  to  terrain  surfaces.  Fast  neighborhood  searches  are  important  for  retrieving  blocks 
of  data  in  simulated  fly-throughs,  and  for  gathering  information  about  neighboring  locations. 
They  also  simplify  line-of-sight  and  shadow  analysis,  contour  generation  and,  in  some  cases, 
hidden  surface  calculations. 

Both  data  compaction  and  clever  organization  help  facilitate  fast  spatial  searches.  The 
regularity  of  gridded  terrain  data  provides  the  best  structure  for  fast  access  with  a  search  time  of 
0(1).  The  more  adaptive  structures  in  relational  and  vector  data  bases  —  such  as  Tactical  Terrain 
Data  --  have  an  inherent  search  time  of  0(/t).  This  is  because  each  record  of  a  relational  struc¬ 
ture  must  be  examined  sequentially  until  the  desired  data  are  found.  Hierarchical  structures  are  a 
compromise,  storing  data  adaptively  while  facilitating  O(logn)  search  time.  To  see  how 
significant  these  orders  of  complexity  can  be,  suppose  that  a  data  base  has  one  million  elements 
to  search.  Searching  for  data  at  a  given  spatial  location  could  take  up  to  one  million  operations 
in  a  relational  structure,  twenty  operations  in  a  hierarchical  structure,  and  one  operation  in  a  ter¬ 
rain  grid. 

1.3  DATA  COMPRESSION  WITHOUT  LOSS  OF  ACCURACY 

Data  compression  facilitates  fast  data  access,  processing,  and  display,  and  enables  simu¬ 
lations  to  employ  larger  gaming  areas.  The  need  for  data  compression  is  best  illustrated  by  look¬ 
ing  at  requirements  for  visualizing  flight  over  a  region.  At  an  altitude  of  200  feet,  a  pilot  can  see 
up  to  30  kilometers  ahead.  With  a  160  degree  field  of  view,  the  area  of  visibility  covers  1254 
square  kilometers.  Using  gridded  data  with  a  post  spacing  of  approximately  100  meters,  this 
corresponds  to  approximately  12,500  posts  or  24,200  triangles.  At  an  altitude  of  2000  feet,  the 
pilot  can  see  up  to  95  kilometers  ahead,  encompassing  3993  square  kilometers  of  terrain.  This 
corresponds  to  approximately  40,000  posts  or  80,000  triangles. 
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Commercial  graphics  systems  can  currently  render  approximately  2000  smooth-shaded 
polygons  in  real  time  (l/30th  of  a  second).  The  two  simulations  described  here  would  therefore 
require  a  compression  ratio  of  12:1  for  the  low  level  flight  and  40:1  for  flight  at  the  higher  alti¬ 
tude.  Furthermore,  it’s  important  to  notice  that  these  numbers  do  not  reflect  the  additional 
polygons  required  to  simulate  cultural  features,  or  map  imagery  on  the  terrain.  Such  typical 
additions  would  further  compound  the  problem.  Therefore,  although  experts  in  the  field  gen¬ 
erally  agree  that  graphics  systems  will  be  able  to  render  more  than  30.000  polygons  in  real  time 
by  the  year  1991,  data  compression  will  still  be  required. 

As  shown  earlier,  subsampling  gridded  data  can  produce  views  which  are  inaccurate, 
and  therefore  inadequate  for  navigation  or  mission  rehearsal  purposes.  It  is  therefore  necessary 
to  find  a  compacting  structure  which  maintains  the  greatest  degree  of  accuracy  possible  at  each 
resolution  level. 

1.4  GRUMMAN  APPROACH 

Our  objective  is  to  develop  terrain  data  structures  which  meet  the  above  requirements. 
Specifically,  we  wish  to  find  a  data  structure  with  the  following  characteristics.  First,  the  data 
must  be  adaptive,  conforming  to  the  unique  characteristics  of  a  piece  of  terrain.  This  data  will 
then  contain  the  smallest  number  of  elements  possible  for  the  required  level  of  accuracy. 
Second,  multiple  levels  of  detail  must  be  readily  available  with  smooth  transitions  between  the 
levels.  Major  features  must  retain  their  positions  and  significant  characteristics  at  all  levels  of 
detail.  Third,  the  data  must  have  an  underlying  structure  which  facilitates  fast  spatial  searches 
for  point  locations,  ranges,  and  neighborhoods.  Fourth,  the  terrain  representation  must  facilitate 
fast  processing  and  display. 

Our  approach  was  to  explore  two  alternative  structures  for  storing  multiple  resolution 
terrain:  quadtrees  and  refined  triangulation  hierarchies.  Both  structures  are  adaptive,  thereby 
storing  greater  detail  for  more  irregular  areas  and  retaining  critical  features  at  even  the  lowest 
resolutions.  We  were  able  to  achieve  10:1  compression  and  better  with  both  structures,  resulting 
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in  little  or  no  loss  of  information  [Tesser89]  [Scarlatos90b].  Both  structures  are  hierarchical, 
facilitating  0(log/t)  search  time.  Hierarchical  organization  of  the  levels  of  detail  —  which  are 
derived  from  one  another  —  also  ensures  consistency  between  resolutions  and  makes  all  levels  of 
detail  immediately  available  at  all  times.  This  is  especially  important  for  dynamic  simulation. 
The  following  sections  describe  our  approach  and  results  in  more  depth. 

2.  QUADTREES 

Quadtrees  have  recently  attracted  much  interest  as  a  means  for  representing  image  and 
cartographic  data.  Part  of  their  appeal  is  their  ability  to  adaptively  represent  2nx2n  data  arrays  in 
hierarchical  levels  of  detail  that  conform  to  a  regular  grid.  Formally,  a  quadtree  is  a  tree  data 
structure  in  which  the  nodes  represent  square  regions  covering  the  original  array.  The  root  node 
covers  the  entire  array.  Each  non-leaf  node  has  exactly  four  children  representing  four  quadrants 
subdividing  the  region  of  the  parent  node.  A  leaf  node  represents  either  a  single  element  in  the 
original  array,  or  a  2kx2k  region  of  the  original  array  which  is  adequately  represented  by  a  sin¬ 
gle  node's  elevation  or  intensity  value. 

Quadtrees  and  their  representations  are  well  documented  in  the  literature  [Samet88], 
One  representation,  the  linear  quadtree  [Gargantini82],  stores  region  data  as  a  collection  of 
nodes  encoded  by  a  sequence  of  directional  codes  also  known  as  quadcodes.  Linear  quadtrees 
are  especially  good  for  spatial  search  because  neighborhood  information  is  intrinsic  to  the 
encoding  of  the  nodes  [Li87],  and  the  path  from  the  root  to  a  node  is  contained  within  that 
node's  quadcode.  Quadcodes  are  concatenations  of  base  4  digits  which  specify  the  location  of 
each  leaf  node  relative  to  the  root  of  the  tree.  Every  digit  in  a  quadcode  corresponds  to  a  qua¬ 
drant  in  the  subdivision  of  a  region.  The  position  of  each  digit  represents  the  level  of  the  tree  at 
which  that  subdivision  was  made.  Thus  the  length  of  a  quadcode  indicates  the  depth  of  a  tree 
node,  and  the  quadcode  itself  provides  the  path  from  the  root  to  that  node.  In  addition  to  the 
quadcode.  each  node  has  an  associated  value  such  as  elevation  or  image  intensity. 
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2.1  BUILDING  QUADTREES 


Quadtree  data  structures  can  be  built  either  top-down  or  bottom-up.  We  build  them  top- 
down  by  examining  the  results  of  a  Laplacian  pyramid,  which  is  built  as  follows  [Tesser89]. 

Starting  with  the  actual  gridded  elevation  data  we  build  a  pyramid  of  data  levels,  each 
level  being  one  quarter  of  the  size  of  the  level  below.  A  pyramid  level  is  created  by  smoothing 
and  subsampling  the  previous  level.  Smoothing  consists  of  convolving  the  original  elements 
with  a  Gaussian  kernel  which  removes  high  frequencies  from  the  data.  The  bottom  of  the  result¬ 
ing  pyramid  is  therefore  the  original  data  array,  and  the  top  is  a  single  element  representing  the 
entire  data  area.  We  then  build  a  second  pyramid  of  data  which  is  smoothed  only.  These  two 
pyramids  are  used  to  find  the  Laplacian  pyramid,  which  is  the  difference  between  the  other  two 
pyramids. 

Finally,  we  build  the  quadtree  by  examining  successive  levels  of  the  pyramid,  starting 
near  the  top.  and  subdividing  nodes  which  need  more  detail.  If  a  node  at  the  curient  level  ade¬ 
quately  represents  the  region  that  it  covers,  then  that  node  becomes  a  leaf  in  the  tree.  We  base 
our  subdivision  decision  on  two  criteria:  the  intrinsic  roughness  of  the  area,  and  extrinsic  factors 
which  focus  on  an  area  of  interest.  Intrinsic  factors  simply  take  into  account  the  fact  that  large 
areas  can  be  used  to  adequately  represent  slowly  varying  terrain.  Extrinsic  factors  consider  areas 
which  have  been  defined  as  being  important  to  an  application,  such  as  an  area  that  a  pilot  may 
be  flying  over  or  the  neighborhood  of  a  target. 

2.2  IMPLEMENTATION  RESULTS 

We  created  a  linear  quadtree  from  the  same  region  of  DTED  used  to  produce  figure  1 . 
as  described  in  section  1.1.  Quadtree  decomposition  took  less  than  8.5  seconds  of  CPU  time  on 
a  VAX  8530,  saving  11,396  posts  in  the  linear  quadtree.  At  display  time,  these  points  were  tri¬ 
angulated  in  less  than  3  CPU  seconds  producing  22,299  triangles  at  the  highest  resolution.  This 
corresponds  to  6:1  data  compression  at  the  highest  level  of  detail.  Visual  comparisons  and  sta¬ 
tistical  analysis  [Tesser89]  verified  the  accuracy  of  this  data  set.  even  at  a  level  producing  10:1 
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compression. 


Figure  2  shows  the  resulting  quadtree  at  three  different  levels  of  detail,  corresponding  to 
approximately  10000,  5000,  and  2500  triangles  respectively.  This  translates  to  13:1.  26:1.  and 
52:1  compression  of  the  original  grid.  Notice  how  well  the  critical  features  are  retained,  as  com¬ 
pared  to  the  subsampled  DTED  grid. 

3.  REFINED  TRIANGULATION  HIERARCHIES 

Triangulated  tessellations  have  long  been  recognized  as  ideal  structures  for  producing 
fast  perspective  views.  Commonly  used  by  geographic  information  systems  (GIS)  and  computer 
image  generation  systems,  triangles  are  ideal  because  all  points  defining  a  facet  are.  by 
definition,  co-planar.  For  this  reason,  other  data  forms  are  often  converted  into  triangles  before 
they  are  rendered  [VonHerzen87]  [Snyder87]. 

Triangulated  irregular  networks  (TINs)  are  triangulated  tessellations  used  to  represent 
terrain  topology.  Formally,  a  TIN  is  a  planar  graph  consisting  of  a  set  of  points  V  and  a  set  of 
non-intersecting  edges  E  which  define  triangular  regions.  These  triangular  regions  completely 
cover  the  convex  hull  containing  all  elements  in  V.  Irregular  triangulations  have  an  advantage  of 
being  highly  adaptive,  allowing  smooth  areas  to  be  represented  by  relatively  few  sample  points 
and  triangles:  rough  or  irregular  areas  may  be  represented  by  as  many  points  as  are  needed  to 
accurately  represent  the  structure.  Irregular  triangulations  may  also  be  made  to  conform  to  irreg¬ 
ular  but  important  boundaries  such  as  ridge  lines,  drainage  systems,  elevation  contours,  coast¬ 
lines.  and  political  boundaries.  The  result  is  a  compact  yet  precise  data  representation. 

Despite  their  advantages.  TINs  do  pose  several  problems.  Each  TIN  represents  a  single 
level  of  detail,  providing  no  guarantee  of  continuity  between  two  different  resolution  TINs  for 
the  same  area.  Furthermore,  spatial  search  on  a  TIN  structure  takes  O(n)  time.  To  overcome 
these  obstacles,  we  developed  the  refined  triangulation  hierararchy. 
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3.1  BUILDING  REFINED  TRIANGULATION  HIERARCHIES 


Refined  triangulation  hierarchies,  first  sketched  in  [Tesser88]  and  described  in  full  in 
[Scarlatos90b],  are  TIN  levels  of  detail  which  are  connected  by  arcs  in  a  hierarchical  fashion. 
Because  each  level  of  detail  is  derived  from  an  earlier  level,  there  is  a  smooth  transition 
between  resolutions,  and  fast  (0(!og  n))  spatial  search  is  facilitated. 

Refined  triangulation  hierarchies  are  built  bottom-up,  starting  with  a  high  resolution  tri¬ 
angulation.  This  high  resolution  triangulation  is  derived  by  finding  the  critical  points  in  a  region 
[Scarlatos90a],  and  then  triangulating  those  points  with  any  number  of  available  algorithms  such 
as  those  in  [Watson84],  [Preparata85],  [Correc87],  [Christensen87]  and  [Scarlatos89].  Every 
other  resolution  level  is  derived  from  the  next  higher  resolution  by  selectively  removing  the 
least  significant  points  and  their  associated  edges,  and  then  retriangulating  resulting  holes  in  the 
mesh.  New  triangles  in  the  lower  resolution  are  then  linked  by  arcs  to  old  triangles  in  the  higher 
resolution,  indicating  overlaps  in  the  different  levels.  This  process  is  repeated  until  the  entire 
area  of  interest  is  represented  by  a  minimal  number  of  triangles. 

3.2  IMPLEMENTATION  RESULTS 

We  created  a  triangle  reduction  data  set  from  the  same  region  of  DTED  used  to  produce 
figure  1.  as  described  in  section  1.1.  The  highest  resolution  data  were  created  by  triangulating 
ridge  and  valley  lines  and  elevation  contours  from  the  DTED  [Scarlatos89],  producing  5933 
points  and  11.774  triangles  in  7.52  CPU  seconds  on  a  VAX  8530.  This  corresponds  to  12:1  data 
compression  at  the  highest  level  of  detail.  Visual  comparisons  and  statistical  analysis 
[Scarlatos90b]  verified  the  accuracy  of  this  data  set.  The  final  refined  triangulation  hierarev  was 
created  in  7:42.29  CPU  minutes  resulting  in  27  resolution  levels. 

Figure  2  shows  the  resulting  refined  triangulation  hierarev  at  three  different  levels  of 
detail,  corresponding  to  approximately  10000,  5000,  and  2500  triangles  respectively.  This 
translates  to  13:1,  26:1,  and  52:1  compression  of  the  original  grid.  Notice  how  well  the  critical 
features  are  retained,  as  compared  to  the  subsampled  DTED  grid. 
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4.  DATA  STRUCTURE  TRADE-OFFS 


In  our  analysis  of  the  two  selected  data  structures,  we  found  that  both  structures  address 
the  requirements  of  the  digital/electronic  terrain  board.  Specifically,  we  found  that  quadtrees 
and  refined  triangulation  hierarchies  have  the  following  advantages. 

First,  both  structures  contain  related  levels  of  detail  in  connected  hierarchies.  This 
ensures  a  smooth  transition  between  resolutions,  allowing  multiple  resolution  views  to  be  gen¬ 
erated  with  ease. 

Second,  fast  spatial  point  and  range  searches  are  facilitated  by  the  hierarchical  struc¬ 
tures.  Neighborhood  searches  may  also  be  done  on  both  structures. 

Third,  both  structures  are  adaptive,  ensuring  that  lower  resolutions  are  missing  only  the 
least  important  data  values.  This  means  that  critical  features  will  retain  their  positions  and  rela¬ 
tive  shapes  at  all  resolution  levels. 

Fourth,  successively  lower  resolutions  in  both  structures  only  eliminate  data  which  are 
least  important  to  the  terrain  topology.  Thus  critical  features  are  retained,  and  only  minimal 
details  are  lost. 

4.1  QUADTREE  ADVANTAGES 

Although  both  structures  are  good  representations  for  terrain  data,  we  did  discover  that 
each  has  its  own  advantages  over  the  other.  The  regular  rectangular  orientation  of  quadtrees,  for 
example,  produces  several  advantages  over  TINs.  One  advantage  is  that  rectangles  aligned  with 
the  rectangular  coordinate  system,  like  partitions  in  a  quadtree,  are  the  simplest  shapes  to  test 
for  intersections.  Consequently,  the  constant  factor  in  the  spatial  search  time  complexity  is  less 
for  quadtrees  than  it  is  for  triangulation  hierarchies.  Linear  quadtrees,  also  by  exploiting  the 
regular  structure,  are  able  to  reduce  the  search  for  neighbors  to  a  simple  indexing  problem  by 
using  quadcodes  [Li87].  Irregular  triangulations  must  rely  on  supplemental  adjacency  informa¬ 
tion  for  neighborhood  searches. 
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Another  advantage  of  quadtrees’  regular  rectangular  structure  is  that  they  are  derived 
naturally  from  commonly  available  gridded  sources  such  as  elevation  matrixes  and  digital 
images.  TINs  must  rely  on  a  separate  algorithm  to  cull  critical  points  from  the  matrixes  to  pro¬ 
duce  points  to  be  triangulated.  Therefore  the  accuracy  of  a  high  resolution  triangulation,  and  all 
related  lower  resolutions,  depends  on  how  critical  points  and  lines  are  selected. 

Finally,  quadtrees  tend  to  occupy  less  disk  space  than  TINs.  Quadcodes  for  a  2nx2n  area 
of  interest  require  2 n  bits  of  data  each.  Each  quadcode  then  needs  an  additional  elevation  value. 
For  our  256x256  test  region,  we  therefore  stored  a  2-byte  quadcode  and  a  2-bvte  elevation  value 
for  each  leaf  in  the  quadtree.  Thus  a  10:1  compression  ratio  on  the  points  results  in  a  5:1 
compression  on  disk.  TINs,  which  must  store  three-dimensional  coordinates  and  adjacency  lists, 
do  not  compress  the  data  as  well  on  disk. 

4.2  REFINED  TRIANGULATION  HIERARCHY  ADVANTAGES 

Refined  triangulation  hierarchies,  on  the  other  hand,  benefit  from  the  flexibility  afforded 
by  irregular  triangulations.  Triangular  facets,  for  example,  are  ideal  for  perspective  image  gen¬ 
eration  and  sensor  simulation.  All  polygon  rendering  hardware  and  software  take  triangles  as 
input,  and  frequently  require  it.  TIN  data  structures  may  therefore  be  rendered  in  real  time. 
Quadtrees  are  not  as  flexible  as  TINs  for  3D  modelling,  requiring  a  triangulation  step  before 
they  may  be  rendered.  Although  triangulation  of  our  test  area  takes  little  more  than  3  CPU 
seconds  on  a  VAX  8530,  this  is  900  times  too  slow  for  real  time  display. 

Another  advantage  of  irregular  triangulations  is  that  they  have  maximum  adaptability. 
This  means  that  points  along  a  ridge  or  valley  line  may  fall  anywhere  on  the  terrain,  with  any 
distance  between  those  points,  and  still  appear  in  the  triangulation.  It  also  means  that  only 
important  points  need  be  saved  in  the  data  base.  Alternatively,  quadtree  regions  must  be  split 
into  regular  quadrants  along  lines  which  have  nothing  to  do  with  the  terrain  characteristics. 
Therefore  some  of  the  points  in  the  quadtree  are  saved  in  a  resolution  level  because  of  location 
within  the  area,  regardless  of  their  topological  significance.  Furthermore,  quadtree  structures  are 
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not  optimal  because  all  child  nodes  must  be  identical  in  size  and  orientation. 

Finally,  TINs  may  be  created  from  varying  data  sources,  including  contour  maps  and 
stereo  imagery,  in  addition  to  the  traditional  grid  formats.  Quadtrees  may  only  be  built  from 
gridded  data. 


5.  CONCLUSIONS 

In  this  paper  we  have  shown  that  two  adaptive  terrain  models  --  quadtrees  and  refined 
triangulation  hierarchies  --  solve  the  problems  presented  by  subsampled  grid  data.  These  struc¬ 
tures  retain  critical  terrain  information  using  minimal  amounts  of  data.  This  ensves  that  impor¬ 
tant  features  will  retain  their  positions  and  general  shapes  at  all  resolution  levels,  without  bog¬ 
ging  the  system  down  with  extraneous  data.  Compression  ratios  of  10:1  and  better  have  been 
achieved  in  our  tests  with  excellent  results.  These  models  also  store  multiple  levels  of  detail 
hierarchically,  providing  smooth  transitions  between  data  levels  and  facilitating  fast  spatial 
search.  Finally,  these  data  structures  may  be  constructed  automatically  within  a  reasonable  time 
frame. 

Our  work  with  these  adaptive  structures  is  not  over  yet.  Currently,  we  are  refining  algo¬ 
rithms  for  building  multiple  resolution  terrain  in  real  time  from  quadtrees  and  refined  triangula¬ 
tion  hierarchies.  We  are  also  exploring  wavs  of  overcoming  the  limitations  listed  above.  Finally, 
we  are  researching  the  applicability  of  these  and  other  hierarchical  structures  to  the  storage  of 
multiple  resolution  cultural  data  for  display  and  analysis. 
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Figure  1:  Full  resolution  DTED  -  131,000  polygons 
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Refined  TIN  -  ’.0.000  polygons 


Quadtree  -  10,000  polygons 


Subsampled  DTED  -  10.000  polygons 


Subsampled  DTED  -  2,500  polygons 


Quadtree  -  2.500  polygons 


Refined  TIN  -  2.500  polygons 


Figure  2:  DTED,  Quadtree,  &  Refined  Triangulation  (TIN)  at  varying  levels  of  detail 
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ABSTRACT 

Functional  Area  Description.  The  goal  of  the  Terrain  Visu¬ 
alization  Functional  Area  is  to  provide  the  Army  C3I  and  training 
communities  with  improved  capabilities  to  rapidly  combine/  effec¬ 
tively  visualize,  and  readily  assimilate  battlefield  terrain, 
environmental,  intelligence  and  maneuver  information.  This  will  be 
accomplished  by  developing  sophisticated  data  fusion  and  3-D  Com¬ 
puter  Image  Generation  (CIG)  technologies;  and  transitioning  these 
capabilities  to  current  and  future  Army  demonstration  and  develop¬ 
ment  programs. 

Principal  Developments  Scheduled.  The  following  describes  the 
principal  hardware/software  developments  scheduled  to  provide  the 
Army  with  this  sophisticated  terrain  visualization  capability. 
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Principal  Development _ 

Install  and  test  real  time  visual  system 
hardware  enhancements  to  CIG  testbed 
Initiate  testbed  demonstration  tailored  to 
user  scenarios 

Complete  user  requirements  analysis 
Demonstrate  CIG  applications  for  DTSS/ASAS 
functionality 

Complete  assessment  of  downsized,  low  cost 
hardware  alternatives 
Complete  design  of  modular  components  for 
DTSS/ASAS  CIG  engine 
Complete  design  of  advanced  3-D  mission 
planning  workstation 


Completion  Date 
3Q  FY89 

4Q  FY89 

FY90 

FY91 

FY92 

FY93 

FY95 


Key  Technologies.  Key  technical  and  developmental  issues  for 
this  effort  fall  into  five  major  areas.  They  are:  1)  real  time 
hardware  developments  and  integration;  2)  digital  terrain  data 
availability  and  integration;  3)  user  interface  and  data  fusion 
capabilities  development;  4)  applications  development,  implementa¬ 
tion  and  demonstration;  and  5)  technology  insertion  into  develop¬ 
mental  systems. 


Hardware  technology  barriers  in  the  areas  of  downsized  high 
throughput  computers,  high  density  storage  media  and  high  resolu¬ 
tion  display  technology  must  be  overcome  to  facilitate  effective 
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utilization  of  CIG  technology  for  tactical  applications.  Expected 
advancements  in  these  technologies  over  the  next  five  years  will 
facilitate  the  development  of  fieldable  CIG  capabilities. 

Digital  terrain  data  availability  is  a  key  issue  for  field 
utilization  of  CIG  technology.  Digital  Terrain  Elevation  Data 
(DTED )  is  the  essential  data  element  for  all  CIG  systems  and  is 
gradually  becoming  available  on  a  world-wide  basis.  Though  Level  I 
Digital  Feature  Analysis  Data  (DFAD )  is  not  sufficient  to  support 
many  tactical  CIG  applications,  plans  for  Defense  Mapping  Agency 
(DMA)  Interim  Terrain  Data  ( ITD) ,  Tactical  Terrain  Data  (TTD)  and 
Air  Force  Project  2851  data,  as  well  as  the  utilization  of  various 
imagery  sources,  will  satisfy  most  CIG  data  requirements. 

Current  CIG  systems  are  highly  interactive  and  require  exten¬ 
sive  operator  interaction,  particularly  in  the  data  base  construc¬ 
tion  and  data  fusion  areas.  Significant  efforts  are  required  to 
develop  robust  data  handling  and  data  fusion  capabilities  that  are 
readily  usable  by  Army  analysts  and  easily  implemented  on  fieldable 
system  architectures. 

Though  many  of  the  currently  available  CIG  systems  operate  at 
real  time  frame  rates  (i.e.,  30  Hz),  the  data  throughput  rates  are 
relatively  low  and  the  resultant  scenes  are  not  particularly  usable 
for  tactical  applications.  However,  many  tactical  applications  do 
not  require  real  time  throughput.  Assessments  will  have  to  be  made 
to  determine  field  Army  scene  generation  requirements,  and  applica¬ 
tions  programs  that  directly  support  tactical  users  will  have  to  be 
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implemented  and  demonstrated.  A  modular,  tactical  CIG  engine  will 
need  to  be  defined  in  order  to  hand  off  the  technology  to  develop¬ 
mental  systems. 

Development  Strategy.  In  order  to  meet  the  objectives  of  the 
Terrain  Visualization  Functional  Area,  the  laboratory  CIG  capabili¬ 
ty  will  be  further  developed  by  providing  support  to  other  DOD 
programs.  The  following  objectives ■ will  be  accomplished  through 
this  systematic  program  support /development  process. 

Provide  advanced  methods  for  3-D  terrain  visualization 

Provide  effective  simulation  techniques  for  smart 
weapons 

Provide  mission  planning/rehearsal  capabilities  in 
denied  areas 

Support  realistic  training  capabilities  in  non-lethal 
environments 

Projected  Capability  (in  the  period  2000-2010) .  Rapid  effec¬ 
tive  Terrain  Visualization  will  be  accomplished  using  modular  and 
compact  3-D  graphics  engines  and  will  be  implemented  as  advanced 
3-D  mission  planning  workstations.  They  will  exploit  high  density 
storage  media  and  high  resolution,  large  area  flat  panel  displays, 
VLSI  circuitry  and  parallel  processing  architectures  for  real  time 
scene  generation.  These  Terrain  Visualization  systems  will  utilize 
DMA,  ITD,  TTD ,  and  Air  Force  Project  2851  data,  reconnaissance 
photography,  IR  and  radar  imagery,  all  source  imagery  and  Image 
Perspective  Transformation  (IPT)  techniques  to  produce  high  quality 
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3-D  perspective  scenes.  Future  CIG  systems  will  provide  data 
throughput  rates  sufficient  to  support  the  detailed  data  sets 
required  by  tactical  users.  This  will  facilitate  implementation  of 
applications  programs  that  support  mission  planning/mission  re¬ 
hearsal,  command  and  control,  maneuver,  IPB  and  special  operations. 
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ABSTRACT 


This  paper  ha3  focused  on  the  process  of  combining  information 
from  disparate  sensors  to  improve  target  tracking  and  classification 
problems  in  an  automatic  target  recognition  (ATR)  system.  There  are 
several  approaches  for  handling  multitarget  multisensor  data  fusion 
(MMDF)  problem.  This  paper  has  also  focused  on  approaches  to 
addressing  the  problems  that  would  relax  the  computational  and  data 
requirements.  This  paper,  in  particular,  has  considered  kinematic  as 
well  as  attribute  data  fusion.  The  kinematic  data  fusion  is 
accomplished  via  a  Kalman  filtering  algorithm  and  the  attribute  data 
fusion  is  performed  by  three  different  fusion  algorithms: 

(1)  Voting,  (2)  Bayesian,  and  (3)  Dempster-Shafer  algorithms.  A 
simulator/evaluator  has  been  developed  to  accomplish  a  non  real-time 
simulation  and  evaluation  of  the  MMDF  algorithms.  The  simulation 
results  have  demonstrated  that  target  tracking  and  classification 
performance  is  greatly  enhanced  by  using  multiple  sensors  operating 
in  different  spectral  regions. 


*  Partially  supported  by  USAF 
Contract  F19628-87-C-0 183 
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1 .  INTRODUCTION 


To  support  the  mission  of  the  strategic  and  tactical  air  control  systems, 
the  air  surveillance  functions  must  provide  a  continuous,  unambiguous,  and 
current  depiction  of  the  airspace  situation  with  respect  to  friendly,  neutral, 
and  hostile  aircraft  or  other  enemy  threats  to  the  personnel  and  other  resources 
in  the  command  centers.  The  following  air  surveillance  capabilities  are 
required  to  successfully  perform  the  mission: 

(1)  detection,  and  identification  of  air  activity  providing  position, 
altitude,  heading,  and  identity  information; 

(2)  continuous  tracking  of  detected  threat  objects,  selected  non-threat, 
and  all  friendly  objects; 

(3)  exchange  of  air  activity  information  between  command  centers, 
tactical  air  control  systems  users,  other  services,  and  the  air 
defense  and  command  control  systems  of  allied  nations; 

(4)  correlation  or  association  of  sensor  tracks  within  the  surveillance 
system  and  with  inputs  from  external  sources; 

(5)  man-machine  interaction  that  will  provide  an  organized  presentation 
of  needed  information  for  the  decision  makers. 


The  advanced  surveillance  systems  must  be  capable  of  providing  decision 
aids  for  the  commanders  who  must  deal  with  a  dense  air  environment;  large 
quantities  of  sensor  data;  multiple  dynamic  sensors;  and  a  dynamic  battle 
environment.  Among  the  currently  fielded  surveillance  assets  that  the  advanced 
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surveillance  systems  must  utilize  are  the  radars,  ESM  sensors,  RWR  sensors,  IFF 


and  JEM  sensors.  Potential  new  surveillance  assets  that  must  be  planned  for 
include  the  MMDF  system  providing  better  target  detection,  tracking,  and 
identification  capabilities. 

The  MMDF  utilizes  a  broad  range  of  frequencies  which  forces  an  enemy  to 
spread  ECM  assets  over  a  large  portion  of  the  electromagnetic  spectrum.  A 
passive  detection  and  tracking  capability  will  support  quiet  operation  when 
needed,  will  assist  in  the  location  of  jamming  aircraft  as  well  as  other 
aircraft  and  cruise  missiles,  and  will  provide  a  capability  of  identifying  non- 
cooperative  aircraft.  The  resulting  fusion  of  on-board  and  remote  surveillance 
sensors  will  fill  in  gaps  in  surveillance  coverage  caused  by  terrain  blockage  as 
well  as  utilize  the  low-altitude  capabilities  of  the  short  range  sensors. 


This  paper  has  described  Sensor  Data  Integration,  Inc.'s  approach  to  the 
correlation  and  fusion  problems  which  arise  when  several  3en3ors  carry  out 
surveillance  over  a  certain  region.  The  overall  objective  is  the  development  of 
an  automated  system  for  detecting,  tracking,  and  identifying  targets  in  a 
multitarget,  multisensor  environment.  Particularly,  this  paper  has  focused  on 
active  and  passive  sensors  a3  previously  mentioned.  The  measurements  from  these 
sensor  3y3tems /subsystems  have  varying  degrees  of  uncertainties.  The 
development  of  the  correlation/fusion  algorithms  accounts  for  the  errors 
associated  with  the  sensor  measurements  for  the  purpose  of  maintaining  an 
updated  combined  track  file  with  a  high  confidence.  Thus,  the  overall  accuracy 
of  the  combined  tracks  is  improved. 
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This  paper  is  organized  as  follows:  an  introduction  to  the  subject  is 
outlined  in  section  1.  The  tracking  and  ID  sensor  systems/subsystems  are 
discussed  in  section  2.  In  section  3,  sensor  data  correlation  techniques  are 
developed;  and  sensor  data  fusion  techniques  are  developed  in  section  4.  The 
simulation  results  are  presented  in  section  5.  A  list  of  references  is 
presented  in  section  6. 

2.  ZD  SENSOR  SYSTEMS /SUBSYSTEMS 

It  is  universally  recognized  that  powerful  sensors  are  extremely  critical 
elements  of  any  battle  management  systems.  As  the  eyes  and  ears  of  the  battle 
managers,  the  sensors  must  provide  necessary  information  about  the  target  of 
interest.  Since  a  target  may  be  characterized  by  many  spectral  frequencies,  a 
number  of  dissimilar  sensors  operating  in  different  spectral  regions  (for 
example,  MM-Wave  radar  (1-10  mm)  and  IR  (8-12  microns))  are  required  for 
detecting  and  identifying  the  targets.  Moreover,  a  number  of  dissimilar  sensors 
operating  in  a  synergistic  manner  are  required  to  detect  the  full  variety  of 
targets  present  in  the  battle  environments. 


The  3ensor  systems /subsystems  considered  in  this  paper  would  provide  two  types 
of  information  about  the  targets: 


(1) 

kinematic 

data 

(position. 

velocity,  and 

acceleration) , 

(2) 

attribute 

data 

(target 

classification 

parameters:  PRF,  PW,  Nature 

(F/H) 
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There  are  two  approaches  to  modeling  the  sensor  systems / subsystems : 

(1)  analytical  approach, 

(2)  statistical  approach. 

This  paper  has  considered  statistical  modeling  of  one  class  of  sensors  which 
include  IFF,  RWR,  and  JEM  sensors.  In  particular,  for  simplicity  this  paper  has 
discussed  only  the  modeling  of  the  IFF  sensor. 


An  IFF  sensor  consists  of  two  subsystems:  an  interrogator  and  a 
transponder.  The  output  of  the  IFF  is  either  "response"  or  "no  response".  The 
"response"  from  the  IFF  is  interpreted  as  friend  (F)  and  "no  response"  means 
either  no  information  or  hostile.  The  statistical  approach  consists  of  two 
steps : 

(1)  generation  of  sensor  declarations, 

(2)  processing  of  sensor  declarations  for  computing  confidence  values. 

Generation  of  IFF  Declarations 

The  IFF  system  includes  two  types  of  errors: 

(1)  stochastic  errors, 

(2)  deterministic  errors. 
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Contributing  factors  of  stochastic  errors: 

•  interference  or  noise, 

•  jamming  (ECM) , 

•  some  trajectory  dependent  errors. 

Contributing  factors  of  fully  deterministic  errors: 

•  lack  of  proper  equipment  and  proper  code, 

•  malfunction  of  the  IFF  system, 

•  hostile  spoofer  with  a  knowledge  of  proper  code, 

•  transponder  not  turned  on  for  any  reason. 


The  IFF  system  declaration  can  be  modeled  as  shown  in  Figure  1. 


Figure  1. 

Modeling  of  IFF  System  Declarations 


The  IFF  system  parameters  are  defined  as  follows: 

Stochastic  parameters ; 

P(F'/F)  *  Probability  of  a  random  response  given  a  friend 
P(F'/H)  =  Probability  of  a  random  response  given  a  hostile 

Deterministic  parameters..;, 

P(F/F')  -  Probability  of  a  consistent  response  given  a  friend 

p(F/H')  -  Probability  of  a  consistent  response  given  a  hostile 

These  parameters  are  used  to  generate  a  sequence  of  IFF  declarations. 
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Now  the  problem  is  to  compute  the  IFF  system  confidence,  P  (F/X)  , 


given  a 


sequence  of  declarations: 

X  =  X1,X2...,Xn;  and  Xi  =  either  F  or  u  for  all  i's.  There  are 
four  basic  3teps  to  compute  the  confidence. 


Step  1  To  compute  the  likelihood  functions 

We  can  compute  the  likelihood  functions  P (X/F)  and  P (X/H) 

as  follows: 

P (X/F)  -P(X/F')  P(F'/F)  +  P(X/H')  P(K' /F)  (1) 

Where  m 

P (X/F' )  -T T  P (Xt/F' ) 
and  i=1 

m _ 

p(x/h'>  -*rr  p(Xi/H') 

i-l 

Similarly,  P(X/H)  is  confuted  as 

P (X/H)  -  P (X/F' )  P(F'/H)  +  P (X/H' )  P(H'/H)  (2) 


Step  Z  To  compute  the  sum 
The  sum  is  calculated  a3 

S  =  P  (X/F)  P  (F)  +  P  (X/H)  P  (H)  (3) 

where  P(F)  and  P (H)  are  known  a  priori  probabilities  based  on  the  force  mix. 


Step  3,  To  compute  the  a  posteriori  probability 

The  a  posteriori  probabilities  (equivalently  confidence  levels),  P(F/X) 
P (U/X)  are  computed  as 


and 
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P (F/X)  =  {  P (X/F)  P (F)  }/  S 


(4) 


and 

P(U/X)  =  {  P(X/U)  P(U)  }/  S  (5) 

Step  1  To  select  the  maximum  a  posteriori  probability 

The  maximum  a  posteriori  probability  is  selected  as  the  IFF  system  confidence 
level  for  the  given  sequence  of  declarations. 

Let  us  explain  the  concept  discussed  above  through  an  example. 

Example 

Assume  that  the  following  are  the  IFF  system  parameter  values : 

P(F'/F)  =  0.96,  P(F'/H)  =  0.001 
P(F/F')  =  0.75,  P(F/H')  =  0.010 

It  is  required  to  compute  the  IFF  system  confidence  level  as  a  function  of  an 
interation  number  when,  for  example,  the  sequence  of  declarations  contains  all 
responses.  It  is  also  assumed  that  P{F)  =  P (H)  =  0.5. 

Solution: 

(1)  X  -  F 

P  (X/F)  =  0.7204,  P  (X/H)  =■  0.0175 
P(F/X)  -  0.9763,  P(U/X)  -  0.0237 

a  A 

(2)  X  -  F,F 

P (X/F)  -  0.5400,  P (X/H)  =  0.0057 
P (F/X)  -  0.9895,  P(U/X)  -  0.0105 
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The  RWR  and  JEM  sensors  have  been  modeled  like  the  IFF  system.  For 
details  of  the  modeling  of  radar  and  ESM  sensor,  the  reader  is  referred  to  [1] . 


3.  MULT I SENSOR  DATA  CORRELATION  TECHNIQUES 
FOR  MULTITARGET  ENVIRONMENT 

This  paper  addresses  the  multisensor  report  to  track  correlation  and 
fusion  problems  which  arises  when  multiple  sensors  carry  out  surveillance  over  a 
certain  region  and  each  sensor  has  its  own  data  processing  unit.  The  question 
is  how  to  decide  whether  two  or  more  reports  from  different  sensor  subsystems 
represent  the  same  target.  The  correlation  refers  to  the  process  of 
associating  or  grouping  together  tracks  as  arising  from  the  same  target.  The 
fusion  refers  to  the  process  of  combining  the  correlated  reports  from  different 
sensor  subsystems  for  the  same  target  into  a  single  track. 


The 

sensor  data 

correlation 

and  correlated 

data  fusion 

constitute 

the 

heart  of 

the  MMDF 

problem. 

The 

process  of  data 

correlation 

and  fusion 

is 

outlined 

in  Figure 

2  . 

Here 

the  correlation  module 

correlates 

the  inputs  from 

all  the  3en3ors  to  the  tracks  in  the  master  track  file.  Then  the  outputs  from 
the  correlation  module  are  provided  as  inputs  to  the  fusion  module  which  uses 
the  correlated  track  data  to  update  the  master  track  file.  This  task  of 
correlation  and  fusion  is  the  subject  of  the  multitarget  multisensor  data  fusion 
problem. 


1.  Sensor  Data  Integration,  Inc.,  "Artificial  Intelligence  Applications  to 
Command,  Control,  &  Communications  Sy3tems/Subsystems , "  Contract  No.  F19628- 
87-C-0183,  February  1988. 
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1  _ 

Fusion  Module 


Figure  2 . 

Measurement  to  Track  Correlation  and  Fusion  Architecture 
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The  key  to  achieving  the  goals  of  real-time  C3  systems  technology  hinges  on 
efficient  techniques  for  real-time  multisensor  data  correlation  and  fusion.  The 
intent  of  this  study  is  to  develop  efficient  and  effective  decision  algorithms 
for  proper  correlation  of  multiple  sensor  data. 


3.1  Sub, optimal  Approach.  PC  Multitarget  Multisensor  Data  Correlation 

In  multitarget  multisensor  correlation  problems,  there  are  two  types  of 
uncertainties:  (1)  uncertainty  associated  with  the  sensor  measurements,  and  (2) 
uncertainty  associated  with  the  origin  of  the  measurements.  There  is  always  some 
uncertainties  associated  with  the  measurements  that  are  generated  by  the 
sensors .  Moreover,  when  there  are  multiple  targets  in  the  same  neighborhood,  it 
is  difficult  to  correlate  multiple  measurements  with  the  multiple  targets.  Thus 
the  multitarget  multisensor  correlation  problem  centers  around  the  measurement 
origin  uncertainty  along  with  the  errors  imported  to  the  sensor  measurements. 
There  are  various  approaches  for  handling  the  multitarget  multisensor 
correlation  problem,  including  optimal  decision,  maximum  likelihood,  and  track- 
split  filter  methods  [2] . 

To  overcome  the  computational  problems  for  real-time  applications, 
suboptimal  Bayesian  approaches  have  been  investigated  in  this  study.  In 
particular,  the  following  3uboptimal  approaches  have  been  considered: 
Probabilistic  Data  Association  (PDA) ,  and  Joint  Probabilistic  Data  Association 
( JPDA)  [1]. 

2.  R.  Carroll,  S.  Chaudhuri,  and  M.  Mayor,  "Multitarget  and  Multisensor 
Tracking,"  A  Five-Day  Short  Course,  The  George 
Washington  University,  D.C.,  June  5-9,  1989. 
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4.  MULTISENSOR  DATA  FUSION  TECHNIQUE  FOR  MULTITARGET  ENVIRONMENT 


The  sensor  systems  considered  in  this  paper  provide  two  types  of 
information  about  the  target: 

(1)  kinematic  data  (position,  and  velocity), 

(2)  attribute  data  (target  nature,  type,  class,  and  scattering  patterns) . 

4.1  Kinematic  Bata,,  Fusion, 

In  the  MMDF  problem  under  consideration,  several  sensors  at  different 
locations  make  measurements  of  the  same  targets.  These  measurements  are 
correlated  and  then  fused  at  a  central  location.  The  3enaor  measurement  errors 
and  times  when  measurements  are  available  for  correlation  and  fusion  greatly 
influence  the  performance  of  the  entire  tracking  and  identification  system. 


The  arrival  times  of  sensor  measurements  at  the  central  location  from 
different  sensors  may  be  synchronized  or  random.  Depending  upon  the  modes  of 
3ensor  measurement  arrival  times,  there  are  different  types  of  measurement 
processing  schemes  [1] . 

In  the  synchronized  data  collection  mode,  there  are  three  types  of  sensor 
measurement  processing  alternatives: 

•  Parallel  Processing, 

•  Sequential  Processing, 
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•  Compressed  Data  Processing. 


In  the  parallel  processing  alternative,  the  sensor  data  are  correlated  in 
parallel  and  then  the  filter  is  updated  with  the  correlated  data  in  parallel.  In 
the  sequential  processing  alternative,  the  correlation  of  measurements  is 
performed  sequentially  and  the  filter  is  updated  with  the  correlated  data  in  a 
sequential  fashion.  In  the  compressed  data  processing  technique,  the  sensor  data 
are  correlated  either  in  parallel  or  sequentially  and  then  the  correlated  data 
are  combined  to  form  an  integrated  data  which  is  then  used  to  update  the  track 
filter.  For  details  of  these  techniques,  the  reader  is  referred  to  [1] . 


4.2  Attribute  Cute.  Fusion 

In  the  attribute  data  fusion,  unlike  sensor  information  are  combined  to  derive 
high  confidence  target  identification.  There  are  various  approaches  to  fuse 
unlike  3ensor  information.  In  particular,  this  paper  has  focused  on  three 
approaches : 

(1)  a  voting  approach  to  multitarget  multisensor  data  fusion, 

(2)  a  Bayesian  approach  to  multitarget  multisensor  data  fusion,  and 

(3)  a  Dempster-Shafer  approach  to  multitarget  multisensor  data  fusion. 

4.2.1  Voting  Approach  La  Multitarget  Multi3ensor  Data  Fusion 

The  voting  approach  can  be  used  in  making  decisions  in  multisensor  data 


fusion  environment.  There  are  two  voting  approaches  considered  in  this  paper: 


(1)  simple  voting  approach, 

(2)  weighted  voting  approach. 

In  the  simple  voting  approach,  the  participating  sensors  are  assumed 
perfect  and  any  declarations  made  by  the  sensors  are  based  without  any 
weighting.  In  this  approach,  one  can  assign  l's  to  those  hypotheses  that  are 
supported  by  the  measurements  and  assign  0's  to  those  not  supported  by  the 
measurements.  The  fusion  consists  of  the  sum  of  the  votes  for  each  target 
candidate.  This  approach  makes  a  decision  on  the  basis  of  which  hypothesis  gets 
the  most  votes.  This  voting  approach  is  simple  to  implement.  This  approach 
does  not  require  sensor  measurement  confidence.  All  that  is  needed  is  a 
database  involving  all  the  hypotheses  which  correlates  the  measurements  with  the 
hypotheses . 

The  weighted  voting  approach  would  require  some  knowledge  about  the 
quality  of  the  sensor  declarations.  This  quality  indication  could  be  in  the  form 
of  confidence  value  as  used  in  the  Bayesian  or  Dempster-Shafer  approach.  When 
sensor  declarations  have  different  confidence  values,  the  weighted  voting 
approach  would  require  a  weight  factor  to  support  a  sensor  measurement. 


4.2.2  Bayesian  Approach  Multitarget  Multisensor  Data  Fusion 

The  Bayesian  approach  to  multitarget  multisensor  data  fusion  is  based  on 
the  maximum  a  posteriori  (MAP)  probability  concept.  Consider  that  Tt  is  the 
target  type  to  be  identified  in  the  presence  of  n  number  of  target  types  in  a 
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given  scenario.  Let  X  be  a  sequence  of  measurements  derived  from  the  same  or 
different  sensors.  Let  us  assume  that  the  measurements  have  been  correlated  for 
the  i-th  target.  Now  the  problem  arises  how  to  combine  the  multiple 
measurements  that  make  up  the  sequence: 


X  -  [xx  ,  x2  ,  •  •  .,  x„  ] 

where  m  is  the  number  of  measurements  to  be  integrated.  The  problem,  therefore, 
is  to  compute  the  confidence,  P(Ti/X),  given  the  sequence  of  declarations  X. 
There  are  four  basic  steps  to  compute  P(Tj_/X).  These  step3  are  identical  with 
the  four  steps  discussed  in  section  2.  However,  Figure  3  3hows  the  Bayesian 


algorithm  flow  diagram  for  sensor  data  fusion  a3  used  in  this  study. 


Likelihood 

Function 

Computation* 


A  Posterior 
Probability 
Computations 


Maximum 
A  Postenor 
Probability 
Selection 


•  P(?/Tj)  z  Likelihood  (unctions 

•  P(T(/f)  z  A  Posterior  Probabilities 

•  max  P  (T|  /  I)  3  Confidence  value  associated  with  the  i  th  target 

i 


Figure  3. 

Bayesian  Algorithm  Flow  Diagram  for  Sensor  Fusion 


4.2.3  Dempster-Shafer  Approach  La  Multitaroet  Multisensor  Data 
Fusion 

The  Dempster-Shafer  theory  of  evidence  yields  similar  results  as  Bayesian 
approach  but  relaxes  some  restrictions  in  the  Bayesian  approach.  The  Dempster- 
Shafer  theory  is  an  extension  of  the  Bayesian  approach  to  include  uncertainty. 
The  Dempster-Shafer  approach  uses  degrees  of  belief  for  representing  uncertainty 
and  allows  one  to  assign  degrees  of  belief  to  subsets  of  hypotheses.  The 
Bayesian  approach  requires  probability  distribution  of  each  element  of  the 
subsets  which  make  up  the  hypotheses . 

In  the  Dempster-Shafer  theory,  the  belief  in  a  proposition  is  represented 
by  credibility  interval  (SPT,  PLS)  where  the  lower  bound  SPT  indicates  the 
degree  to  which  the  evidence  supports  the  hypothesis  and  the  upper  bound  PLS 
indicates  the  degree  to  which  the  evidence  fails  to  refute  the  hypothesis.  Each 
credibility  interval  (SPT,  PLS)  is  a  subinterval  of  the  probability  interval 
(0,  1)  .  The  difference  between  the  upper  and  lower  bounds  implies  the  residual 
ignorance.  Complete  ignorance  is  represented  by  interval  (0,  1) .  Figure  4 
outlines  the  Dempster-Shafer  algorithm  flow  diagram  for  sensor  data  fusion. 
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Figure  4. 

Dempster  -  Shafer  Algorithm  Flow  Diagram  for  Sensor  Data  Fusion 
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5. 


SIMULATION  RESULTS 


Let  us  consider  an  example  that  demonstrates  how  target  tracking  can  be  improved 
by  using  the  multisensor  fusion.  Figure  5  depicts  two  targets  in  a  horizontal 
plane  starting  at  100  nmi  from  multiple  sensors  which  are  colocated  at  the 
origin  of  the  assumed  coordinate  system.  The  two  targets  are  moving  in  straight 
line  trajectories  with  azimuthal  angles  of  15  and  30  degrees  respectively.  3oth 
the  targets  are  maintaining  a  constant  speed  of  500  knots. 

The  simulator/evaluator  ha3  been  used  to  generate  the  following: 

(1)  target  trajectories, 

(2)  radar,  ESM,  IFF,  RWR,  and  JEM  measurements  and  associated 
variances, 

(3)  target  state  estimation  error  standard  deviations  (x  and 
y  components) , 

(4)  confidence  levels  associated  with  the  target  ID's. 


STATE  ESTIMATION  ERROR  STANOARO  DEVIATION  (THETA  •  30  OEGfTEES)  STATE  ESTIMATION  ERROR  STANDARD  DEVIATION  (THETA  m  JO  DECREES) 


Tive  (sicoNot) 


Figure  5. 

Trajectories  for  Two  Targets 
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In  the  Phase  I  final  report  [1],  a  few  test  cases  are  presented  to 


demonstrate  the  performance  of  the  MMDF  algorithms  for  target  tracking  and 
identification.  Test  cases  have  been  formulated  to  illustrate  the  benefit  of 
the  MMDF  algorithms.  Only  one  test  case  is  outlined  in  this  paper. 

The  scenario  for  this  analysis  includes  one  friendly  and  one  hostile 
aircraft.  The  known  key  parameters  of  these  targets  are  given  in  the  following 
database  shown  in  Table  1 . 


Table  1. 


Target  Parameters 

Targets 

Nature 

JEM 

RWR 

2 

RCS  (m  ) 

F-l 

F 

3 

5 

3 

H-l 

H 

1 

3 

2 

The  test  case  has  considered  F-l  and  H-l  aircraft.  The  radar  and  ESM 
sensors  have  been  deployed  for  tracking  the  targets.  A  host  of  other  sensors 
are  also  deployed  to  accurately  identify  the  targets.  For  simplicity,  the 
tracking  results  of  only  one  target  are  outlined  in  this  report.  Figure  6 
illustrates  tracker  performance  along  x  and  y  axes  respectively  for  the  30 
degree  target.  This  figure  indicates  that  the  tracking  performance  is  agreeably 
better  when  the  targets  are  tracked  by  both  the  radar  and  the  EMS  sensor 
combined  that  when  tracked  by  either  radar  or  EMS  sensor.  These  results 
demonstrate  that  tracking  performance  can  be  greatly  enhanced  in  the  multisensor 
fusion  environments. 
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2  ANCLE  •  +15  OEGREES 


Figure  . 6 

Tracker  Performance  Results 
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The  following  in  Table  2  are  the  correlated  sensor  declarations  associated  with 
the  tracked  target  at  a  particular  time  t-1  second: 


Table  2 . 


Declaration 

Declarations 

Confidence 

Confidence 

Sensors 

IFF 

Not  Friend 

0.95 

RWR 

R  Class  3 

0.98 

JEM 

J  Class  1 

- -  . 

0.98 

FPC 

Outside 

0.86 

2 

RCS 

2  (m  ) 

0.80 

Assuming  uniform  force  mix  (that  is  P(F)  =  P(H)  =  1/2),  Bayesian 

approach  yields  the  following  confidence  values  in  Table  3. 


Table  3 . 


Confidence  - 

Targets  (A  )  P (A  /x) 

i  i 

Confidences 

F  -  1 

0.0010 

H  -  1 

0.9997 
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ABSTRACT 


This  paper  describes  a  multidisplay,  multifunction,  multiprocessor 
approach  to  a  cockpit  simulation  environment  that  allows  for  several 
simultaneous  display  formats  to  operate  in  parallel.  The  approach 
produced  a  state-of-the-art  cockpit  simulation  that  is  low  cost, 
distributed  across  several  processors,  and  is  readily  expandible  to 
new  situations.  Additional  computing  capability  can  be  added  through 
the  uses  of  a  sophisticated  message  handling  scheme.  The  system  is 
readily  customizable  to  most  display  formats  that  are  currently  in 
existing  aircraft,  and  new  formats  can  readily  be  added  as  needed. 
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Introduction 


A  single-seat  attack  aircraft  simulation  facility  that  would  allow 
human  factors  performance  evaluations  in  a  real-time,  multidisplay, 
multifunction  environment  was  required  as  part  of  the  Intelligent  Air 
Attack  System  (IAAS)  development (ref  1) .  A  testing  environment  was 
developed  around  a  Silicon  Graphics  IRIS  graphics  workstation  that 
allows  investigators  to  evaluate  task  performance  when  a  pilot  is 
required  to  perform  several  simultaneous  tasks.  The  environment 
simulates  a  detailed  state-of-the-art  cockpit  that  contains  an  out-of- 
cockpit  view,  a  heads-up  display  (HUD) ,  three  multipurpose  displays 
with  operational  push  buttons,  a  multipurpose  data  entry  control 
panel,  a  throttle  and  stick  with  multifunction  control  switches,  and 
numerous  cockpit  switches.  After  its  initial  success  as  the  testing 
facility  of  the  Intelligent  Air  Attack  System,  the  facility  is  now 
being  reconfigured  to  other  experiments  involving  pilot  performance, 
display  evaluations,  and  work  load  studies. 

The  cockpit  displays  supplied  on  the  CRT  of  a  Silicon  Graphics 

IRIS  workstation  are  animated  at  a  refresh  rate  of  10  Hz.  A 

touchscreen  overlays  the  display  to  allow  interaction  with 

graphically  generated  push  buttons.  The  simulation  supplies  the  pilot 

with  real-time  animated  displays  which  represent  the  information  he 

(ref  1)  Intelligent  Air  Attack  System  (IAAS) :  Project  and  Product 
Definition,  by  Judith  Lind,  China  Lake,  Calif.,  NWC,  April  1988,  NWC 
TM  6161  (  Unclassified) . 
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would  receive  from  onboard  sensors,  system  databases  and  avionics (see 
Figure  2.).  Multiprocessor  communication  is  maintained  between  the 
IRIS,  two  Zenith  80386  computers  and  an  IBM  PC  through  the  use  of  an 
ethernet  network  using  TCP/IP  protocol.  Communication  to  and  from  the 
non-IRIS  related  cockpit  controls  and  switches  use  the  ethernet,  the 
mouse  port  and  an  additional  RS-232  port. 

Description 

The  physical  cockpit  environment  is  designed  to  give  the  pilot  the 
sense  and  feel  of  a  real  cockpit.  The  NWC  physical  cockpit  mockup 
consists  of  a  4  x  4  x  12-foot  framed  shell  covered  with  masonite,  an 
aircraft  ejection  seat,  an  F/A-18  throttle  and  stick  with 
multifunction  control  switches,  a  set  of  fixed-function  panel 
switches,  a  19  inch  Hitachi  monitor  interfaced  to  the  IRIS,  and  foam 
core  mockups  of  other  standard1  cockpit  switches  and  panels. 

The  simulation  software  is  categorized  into  six  major 
components:  (1)  the  main  program, (2)  the  aircraft  simulation, (3)  the 
external  world  models,  (4)  the  avionics  models, (5)  the  message  handler 
and  (6)  the  display  control  system.  Each  component  is  described 
below. 
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The  NWC  simulation  is  a  version  of  the  Silicon  Graphics  flight 
program  that  has  been  radically  modified.  The  primary  program  was 
reduced  to  one  page  of  function  calls,  and  new  functions  needed  by  our 
requirements  were  added.  The  program  has  grown  from  the  original  170 
kilobytes  of  "C"  language  source  code  to  over  1.5  megabytes  of  source 
code. 

The  airframe  simulation  is  a  six-degree-of-freedom  simulation, 
that  can  fly  several  aircraft  models  simultaneously.  The  flight 
characteristics  of  each  aircraft  are  defined  by  a  set  of  parameters 
that  are  carried  internally  in  the  aircraft  data  record.  The  primary 
aircraft  has  the  characteristics  of  a  subsonic,  heavily-loaded  single¬ 
seat  attack  aircraft. 

As  the  need  for  other  types  of  dynamic  models  became  apparent, 
it  was  noted  that  many  of  them  could  be  inserted  into  the  simulation 
as  "aircraft”  and  manipulated  as  such.  Ships,  missiles,  as  well  as 
tanks  and  armored  personnel  carriers  were  made  available,  each  with 
its  own  "flight  characteristics"  ranging  from  "very  slow  and  sluggish" 
to  "very  fast  and  maneuverable". 

The  IRIS  graphics  world  model  contains  sky,  ocean,  land,  and  any 
other  objects  such  as  ships  and  missiles  which  are  located  visually 
outside  of  the  cockpit.  The  graphics  processors  manipulate  the  imagery 
and  present  the  pilot  with  the  out-of-cockpit  views  that  result  from 
manipulating  the  aircraft. 
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Each  ship  model  derives  its  identity,  speed  and  location  from  the 
scenario  files  that  are  loaded  at  the  beginning  of  the  program,  and 
then  independently  simulates  the  behavior  of  that  ship  for  the 
remainder  of  the  simulation  run.  Each  ship  model  has  radar  emissions, 
and  determines  the  current  mode  of  operation  that  is  compatible  with 
its  weapons  and  sensors.  The  radar-warning-receiver  (RWR)  model  reads 
each  ship  model  for  emitter  information. 

When  the  missile  simulation  is  started  as  a  result  of  a  ship 
initiating  a  launch,  the  missile  derives  its  flight  characteristics 
from  the  ship's  database,  and  then  flies  out  toward  the  aircraft  using 
a  lead-intercept  algorithm.  The  intercept  algorithm  is  designed  to 
intercept  and  detonate  the  missile  when  it  reaches  its  closest 
proximity  to  the  target.  Missiles  can  be  defeated  if:  (1)  the  aircraft 
drops  below  the  radar  horizon,  (2)  the  missile  falls  into  the  sea,  or 
(3)  an  active  expendible  countermeasure  leads  it  astray.  Due  to  the 
complexity  of  the  algorithms  and  the  fact  that  realistic  update  rates 
must  be  maintained,  only  one  missile  at  a  time  is  currently  allowed. 
Certain  display  formats  are  frozen  during  the  later  segments  of  a 
missile  flyout  and  only  those  displays  essential  to  flying  the  mission 
continue  to  be  updated  at  the  normal  update  rate. 
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Models  of  expendible  electronic  warfare  countermeasures  (flares, 
chaff,  jammers,  etc)  are  designed  to  have  a  specified  life  usually 
measured  in  seconds,  and  lose  their  effectiveness  exponentially  over 
time.  Only  the  judicious  use  of  the  countermeasures  combined  with  the 
correct  timing  of  the  deployment  will  result  in  an  effective  defense 
against  the  missile  threats. 

The  simulation's  radar  model  supplies  the  positional  data 
necessary  for  the  radarscope.  The  NWC  simulation's  radar  model  has  a 
sea-search  mode  and  a  sea-surface-track  mode.  Track-while-scan 
capability  is  not  currently  implemented.  The  tracking  of  a  surface 
target  results  in  the  loss  of  radarscope  imagery  for  any  other 
contact.  Figure  2  contains  a  sample  of  the  radarscope  format. 

Part  of  the  war-at-sea  scenario  called  for  processing  of  Inverse 
Sythetic  Aperture  Radar  (ISAR)  imagery.  The  size  of  the  database 
precluded  its  existing  in  the  main  simulation.  Network  messages  were 
generated  by  the  radar  model,  which  instructed  another  processor  on 
the  network  to  process  the  correct  files  and  return  the  data  to  the 
display  processors  for  pilot  evaluation.  The  display  format  of  the 
returned  data  is  in  Figure  2 . 


The  Stores  Management  System  currently  manages  Harpoon  missiles. 
Weapon  programming  parameters  are  loaded  through  the  simulated 
keyboard  entry  device  known  as  the  upfront  control  panel,  while  other 
weapon  parameters  can  be  downloaded  using  the  bezel  buttons  on  the  SMS 
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display.  A  hardwired  Master  Arm  switch  and  stick  switches  must  be 
activated  to  "release"  the  weapons.  See  Figure  2  for  the  display 
format. 

The  RWR  uses  the  radar  emission  data  from  ships,  and  generates 
outputs  that  an  actual  RWR  would  present  to  the  pilot.  No  true 
emission  processing  is  done.  However  the  model  does  intentionally 
introduce  azimuth  ambiguity  and  some  random  spurious  signals  to  make 
its  output  more  realistic.  An  RWR  display  format  is  displayed  in  the 
window  area  (See  Figure  1.). 

The  Missile  Detector  System  (MDS)  simulates  a  short-range  doppler 
radar  system  capable  of  detecting  in-the-air  missiles.  The  information 
is  sent  to  another  peripheral  program  which  determines  the  threat 
identification  and  how  best  to  defeat  it.  The  MDS  model  records  only 
those  threats  that  are  closing  on  the  aircraft.  An  MDS  display  format 
is  used  to  present  information  to  the  pilot,  and  is  overlayed  onto  the 
HUD. 


The  forward  looking  infrared  (FLIR)  model  is  simulated  by  creating 
images  of  vehicles  colored  to  represent  the  object  as  it  would  look  in 
infrared.  The  images  are  then  filtered  to  generate  various  amounts  of 
noise,  and  are  made  available  on  the  FLIR  display  format. 

The  systems  message  handler  is  an  interface  between  the 
simulation  and  the  other  computers  on  the  ethernet  network  and 
performs  two  major  tasks,  synchronizing  of  the  startup  of  the  programs 
on  the  other  processors,  and  passing  messages  between  the  various 
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processors.  Individual  messages  are  passed  to  and  from  the  various 
computers  on  the  system  using  any  of  the  eighty  predefined  messages. 
The  messages  are  directly  vectored  to  the  receiving  computer's 
processing  software  by  their  message  number  where  the  data  is 
extracted  and  processed. 

The  display  control  system  is  a  set  of  functions  designed  to 
control  and  display  any  of  the  display  formats  currently  available. 
This  system  consists  of  display  management  functions,  display 
generation  and  updating  functions,  and  bezel  button  functions. 

Examples  of  formats  are  in  Figure  2. 

The  display  management  software  manages  two  sets  of  data,  one 
representing  the  logical  displays  and  the  other  representing  the 
physical  displays.  Each  logical  display  represents  an  active  display 
format  available  for  display,  and  controls  the  manipulation  of  the 
format.  The  physical  display  controls  which  logical  displays  are 
visible  and  where  they  are  located  on  the  CRT  (see  Figure  1.). 

The  logical  display  management  software  is  centered  around  logical 
display  structures  which  contain  the  information  that  is  pertinent  to 
the  operation  of  a  specific  logical  display.  Logical  displays  contain; 
the  graphics  objects  identifiers,  pointers  to  the  functions  associated 
with  the  display,  and  several  flag  variables. 

The  physical  display  functions  control  the  mapping  of  the  logical 
displays  to  the  physical  display  zones  on  the  surface  of  the  monitor. 
There  is  currently  one  physical  display  structure  for  each  of  the  six 
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graphics  zones  on  the  monitor.  Logical  displays  can  be  reassigned  to 
any  of  the  three  multifunction  displays  at  any  time,  and  in  any  order. 
Experience  has  shown  that  it  is  unwise  to  map  a  logical  display  to 
more  than  one  physical  display  zone  since  only  one  of  the  physical 
display  zones  will  be  redrawn  after  an  update.  Except  for  the  menu 
display,  only  one  of  each  avionics  display  format  is  allowed. 

Different  formats  have  different  refresh  requirements.  Update 
rates  vary  from  10  Hz  to  no  update  needed.  Updates  to  the  CRT  are 
tightly  controlled  since  the  graphics  refreshes  are  time  consuming. 

The  individual  display  formats  consist  of  foreground  and  background 
images.  Writemasking,  a  graphics  process,  is  used  to  protect  static 
portions  of  a  display  from  being  erased  during  an  animation  sequence. 

Each  display  format  has  a  description  file  that  is  in  the  form 
of  a  header  file  and  a  format  template  file  .  The  header  file  contains 
arrays  of  button  labels  and  an  array  of  bezel  button  functions 
allowing  the  programmers  to  change  bezel  functions  and  labels  easily 
without  searching  through  code.  The  format  template  file  contains 
functions  which  perform  the  following:  define  the  labels,  define  the 
foreground  object,  define  the  background  object,  define  the  update 
function,  and  refine  all  bezel  button  functions  that  are  unique  to  the 
display  format. 


269 


The  NWC  simulation  has  nearly  unlimited  display  format 
capabilities.  New  display  formats  are  added  to  the  system  by 
generating  the  model  to  create  the  data,  defining  the  display  format 
that  meets  the  template,  and  then  defining  what  its  logical  display 
format  parameters  are.  Once  built,  the  display  is  made  available  by 
activating  it  from  some  other  existing  format,  or  causing  it  to  be 
mapped  to  a  display  zone  when  the  simulation  starts. 

Control  functions  operated  via  the  bezel  buttons  are  executed 
directly  from  the  logical  display  structure.  A  button  press  executes 
the  function  defined  in  the  logical-display-button  array.  Changing  the 
functionality  of  the  button  is  as  simple  as  loading  another  function 
into  that  element  of  the  array. 
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Conclusions 


The  Intelligent  Air  Attack  System  (IAAS)  was  the  first  project  to 
fully  utilize  the  cockpit  simulation.  Decision  aids,  task  management, 
and  display  managment  was  supplied  by  the  peripheral  programs  and 
presented  to  the  pilots  in  a  war-at-sea  scenario.  The  cockpit  allowed 
pilots  to  interact  with  decision  aids,  and  task  orientated  displays, 
while  attempting  to  fly  the  aircraft,  launch  weapons,  and  counter 
hostile  missiles.  Two  sets  of  simulations  were  reguired  for  the 
evaluations,  a  baseline  "F/A-18  like"  cockpit  (ref  2)  and  the  IAAS 
cockpit (ref  3).  Display  formats  were  developed  for  both  .  IAAS  has 
proven  itself  to  be  a  viable  technology  that  could  prove  itself 
valuable  in  the  1990's. 

The  cockpit  simulator  is  a  low  cost  approach  that  can  meet  the 
needs  of  many  projects.  Future  expansion  of  the  system  will  include 
the  addition  of  other  IRIS  graphics  processors  and  displays  to  widen 
the  field  of  view,  and  parallel  processing  capability  to  increase  the 
fidelity  of  the  models. 


(ref  2)  McDonnel  Douglas  Corp.  F/A-18  Avionics  Display  Format 
Diagrams, 

St  Louis,  Mo.,  17  Mar  1980  (  Unclassified) 

(ref  3)  Intelligent  Air  Attack  System:  Display  Format  Diagrams  by 
Judith  Lind,  China  Lake,  Calif.,  NWC,  April  1988,  NWC  TM6207 
(Unclassified) 
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Top  Menu  Format 


Tactical  Situation  Format 


Radar  Track  f'bde  Format 


Figure  2.  Example  Display  Formats  used  in  the  IAAS  Project. 
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ABSTRACT 


The  SANDTABLE  Project  was  conceived  to  show  how  the  US  Army  Communications-Electronics  Command 
(CECOM)  sensor  systems  are  used,  what  they  do,  where  they  are  employed,  what  enemy  force  they  detect 
and  how  they  interface  with  all  command  and  control  systems  on  the  battlefield.  SANDTABLE  utilizes 
a  computer  controlled  system  to  display  maps  and  imagery  and  to  perform  analysis  of  multi-source 
data.  Graphics  depicting  locations,  events,  sensors,  mission  plans  and  lines  of  communication  can 
be  presented  over  the  maps  and  imagery  on  a  high  resolution  color  monitor.  By  utilizing  the  Digital 
Terrain  Elevation  Data  (DTED),  terrain  analysis  such  as  determining  the  terrain  elevation,  slope 
profiles,  three  dimensional  terrain  view,  terrain  masking  and  radial  line  of  sight  can  be  per¬ 
formed.  Engineer  Topographic  Laboratories’  three  dimensional  video  terrain  and  models  of  sensors 
can  be  integrated  into  the  SANDTABLE  computer  system  and  displayed  on  the  color  monitor. 


1.0  INTRODUCTION 

1.1  SANDTABLE  Tasking 

The  SANDTABLE  project  was  originated  from  the  tasking  given  by  CECOM  Commanding  General,  Major 
General  Billy  M.  Thomas.  His  tasking  was  to  build  a  huge  SANDTABLE  with  intelligence/ 
electronic  warfare  rigs  deployed  over  a  Corps  sector  showing  all  the  present  and  future  sensor 
systems  and  lines  of  communication. 

1.2  SANDTABLE  Implementation 

In  order  to  fulfill  the  SANDTABLE  tasking,  an  approximately  1000  square  feet  of  SANDTABLE  and 
sensor  models  have  to  be  built.  Due  to  the  physical  size  of  this  huge  SANDTABLE  and  small  sensor 
models  constraint,  a  different  approach  utilizes  the  computer  controlled  system  has  been  used.  The 
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computer  controlled  system  is  capable  of  displaying  maps  and  imagery  and  performing  terrain 
analysis.  Thus,  by  utilizing  the  computer  controlled  system,  the  physical  SANDTABLE  with  sensor 
systems  can  be  easily  simulated.  A  one  hour  videotape  version  which  describes  the  electronic 
battlefield  is  being  used  in  lieu  of  the  physical  SANDTABLE. 

2.0  ELECTRONIC  BATTLEFIELD  DISPLAY 

2.1  Computer  Controlled  System 

The  computer  controlled  system  consists  of  the  following  hardware  components: 

a.  Personal  computer  with  16  bits  microprocessor  and  math  co-processor,  graphic  processor, 
keyboard,  mouse  and  a  high  resolution  color  computer  monitor. 

b.  Videodisc  player 

c.  High  resolution  color  TV  monitor 

d.  CD-ROM  player 

e.  Optical  disc  recorder/player 

f.  Video  camera 

g.  Color  printer 

The  computer  system  is  capable  of  controlling  and  accessing  information  from  the  videodisc  player, 
CD-ROM  player  and  the  optical  disc  recorder/player.  It  can  also  perform  information  management, 
terrain  analysis,  measurement  and  display. 

The  computer  system  utilizes  the  videodisc  player  to  display  videodisc  maps  on  the  high 
resolution  color  TV  monitor.  Graphical  characters,  symbols  or  ICONS  can  be  overlayed  on  the  map 
background.  The  computer  system  can  retrieve  Digital  Terrain  Elevation  Data  (DTED)  from  the  CD-ROM 
player  and  perform  terrain  analysis.  By  using  the  optical  disc  recorder  and  video  camera,  still 
photographs  can  be  recorded.  Three  dimensional  static  and  dynamic  models  of  sensors  and  targets 
placed  on  terrain  images,  and  a  dynamic  fly  by  of  models  can  also  be  recorded  on  the  optical  disc 
and  played  back  on  the  TV  monitor.  Hard  copy  of  text  and  graphics  can  be  printed  out  on  the  color 
printer. 
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The  hardware  components  are  low  cost  and  commercially  available  products.  Videodisc  maps  are 
widely  available  through  the  Defense  Mapping  Agency  (DMA),  COINS,  Joint  National  Intelligence  Dev. 
Staff  (JNDDS),  BDM,  Computer  Corporation  of  America  (CCA),  Interactive  Television  Company  (ITC), 
Routing  Technology  Services,  Inc.  (RTSI),  Defense  Advanced  Research  Projects  Agency  (DARPA),  Sandia 
National  Laboratory  (SNL), System  Control  Technology  (SCT),  and  Institute  for  -  Defense  Analysis 
(IDA).  The  above  list  is  not  all  inclusive,  there  are  other  agencies  not  listed  above  which  also 
produced  videodisc  maps. 

22  Electronic  Battlefield  Display 

To  generate  a  battlefield  condition,  map  of  Europe  is  displayed  on  the  monitor.  Using  the 
graphical  capability  of  the  system,  the  boundary  of  7th  Corps  is  drawn.  The  Corps  area  is  further 
broken  down  into  three  divisions,  and  three  brigades  in  a  division.  Graphical  character 
representation  for  the  CECOM  sensor  and  jammer  systems,  and  the  enemy  systems  are  drawn  and  depicted 
on  the  map  background.  Systems  information  descriptions,  capabilities,  drawings  and  pictures  are 
recorded  and  stored  on  the  optical  disc  as  collateral  data.  Communications  flow  from  the  sensor 
systems  to  the  commander  is  graphically  represented  on  the  map  background.  The  description  of  the 
Army  tactical  command  and  control  system  architecture  is  recorded  and  stored  on  the  optical  disc. 
The  optical  disc  is  used  as  a  storage  means  for  the  collateral  data  and  retrieved  by  the  computer 
and  displayed  on  the  TV  monitor. 

The  Engineer  Topographic  Laboratories’  three  dimensional  model  of  sensors,  terrain  images  and 
dynamic  fly  by  of  sensors  are  recorded  as  collateral  data  on  the  optical  disc.  To  retrieve  these 
collateral  data,  mouse  cursor  is  placed  on  the  specific  geographic  location  or  sensor  systems 
graphical  character  on  the  map  background.  The  computer  searches  the  collateral  database  for  data 
associated  with  the  coordinates  near  the  cursor  location  and  displays  user  selectable  item  menu  on 
the  computer  display  monitor.  The  collateral  data  will  then  be  displayed  on  the  TV  monitor  upon 
user  selection  for  displays. 

The  Digital  Terrain  Elevation  Data  (DTED)  is  used  to  perform  terrain  analysis  such  as  radar 


terrain  masking,  radial  ime  of  sight  calculation,  oblique  terrain  view,  tinted  elevation  map 
display  and  terrain  slope  display.  The  radar  terrain  masking  function  provides  a  graphic  display  of 
the  radar  terrain  mask  from  a  marked  point  by  inputting  radar’s  antenna  height,  range  and  ray 
spacing  in  degrees.  The  radial  line  of  sight  function  provides  a  graphic  display  of  the  line  of 
sight  in  all  directions  from  a  marked  point  by  inputting  observer’s-  height,  viewing  range  and  ray 
spacing  in  degrees.  The  oblique  terrain  views  function  provides  a  three  dimensional  terrain  graphic 
on  the  computer  display  monitor  by  marking  the  observer  and  target  locations.  The  tinted  elevation 
map  function  overlays  the  map  with  colors  representing  elevation  levels.  The  terrain  slope  function 
overlays  the  map  with  a  pattern  of  colors  representing  the  slope  of  terrain. 

In  order  to  simulate  a  battlefield  condition  and  to  display  the  results  of  battlefield  changes, 
the  real  time  updating  function  is  being  used.  The  graphical  representation  of  the  sensor  system 
with  geographic  location  on  the  map  background  can  be  updated  from  a  remote  system  via  the  RS  232 
port  to  the  computer.  The  updated  file  contains  changes  in  movement  and  speed  of  the  sensor  system 
indicating  battlefield  changes.  The  computer  system  receives  the  real  time  updating  information  and 
displays  position  movements  on  the  TV  monitor. 

3.0  CONCLUSION 

The  SANDTABLE  project  utilizes  a  computer  controlled  video  presentation  to  describe  the 
extensive  use  of  electronics  on  the  modern  battlefield.  The  SANDTABLE  computer  system  enables  the 
users  to  perform  mission  planning,  information  management,  terrain  analysis,  measurement, 
information  display  and  real  time  event  updating.  The  SANDTABLE  computer  system  can  be  used  for 
different  purposes  and  applications  such  as  presentation,  simulation  or  test  bed. 
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Tar rain  and  Target  Signature  Simulation 

Dr.  John  H.  Lewis 
Hughes  Aircraft  Company 
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Long  Beach,  California  90810-0399 

ABSTRACT 

Just  as  significant  advancements  in  imaging  weapon  systems 
have  expanded  battlefield  tactical  concepts,  this  performance 
improvement  has  dictated  a  need  for  advancements  in  test  and  evalu¬ 
ation  techniques.  The  basis  for  both  expanded  weapon  system 
employment  and  more  capable  development  tools  is  technology  driven. 
Today  in  the  growing  field  of  terrain,  target,  and  sensor  simula¬ 
tion,  Computer  Image  Generation  (CIG)  is  an  important  tool  to 
improve  and  hasten  design  development,  test  and  evaluation,  and 
tactics  development. 

Simulation  of  multi-band  imaging  infrared  seekers  for  the 
tactical  environment  is  under  way  at  a  number  of  laboratories.  A 
system  approach  based  on  photo  and  sensor  image  inpats,  which  is 
derived  from  first  principals  and  produces  real  time  ambient  condi¬ 
tion  specifiable  infrared  imagery,  will  be  described.  This 
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approach  has  been  demonstrated  and  is  being  integrated  for  both 
tactical  and  strategic  infrared  imaging  in  a  hardware-in-the-loop 
configuration . 

Advanced  processing  techniques,  including  a  number  of  graphics 
structures  (polygons,  fractals,  voxels) ,  are  incorporated  as  well 
as  two  and  three-dimensional  image  processing. 
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Image  Generation  -  VI CAM 
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ABSTRACT 

Over  a  decade  ago,  Honeywell  pioneered  the  work  in  photo-based 
image  generation  (in  the  IR  and  visual  domains)  for  missile  evalua¬ 
tion  and  training  systems.  This  technology  is  now  being  applied  to 
mission  planner/rehearsal  systems  and  aircraft  cockpit  instrumenta¬ 
tion.  Large,  accurate,  ultra-high  resolution  photo-quality,  multi- 
spectral  displays  at  high  update  rates  from  a  common  database 
appear  to  be  requirements  for  both  mission  rehearsal  and 
Digital/Electronic  Terrain  Board  (D/ETB)  systems. 

Creating  real-time  computer  generated  images  of  terrain  and 
man-made  objects  of  sufficient  fidelity  and  accuracy  to  serve  as  an 
ATR  test  system  is  the  difficult  problem  for  D/ETB.  One  solution 
is  to  geometrically  and  temperature/emissivity  model  the  terrain/ 
objects.  This  method  has  numerous  problems,  including  the  verifi¬ 
cation  of  the  accuracy  of  the  model.  Honeywell's  approach  uses 
real  images  manipulated  digitally  by  empirical  and  model  data  to 
yield  extremely  high  quality  3D  perspective  views.  These  views  may 
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be  manipulated  to  simulate  the  desired  moving  sensor/platform/ 
target  situation.  Furthermore,  the  resolution  and  sensor  type  are 
limited  only  by  the  input  imagery  provided.  Atmospheric  and  sensor 
effects  are  also  simulated  digitally.. 

Honeywell's  solution  for  high  fidelity  aircraft  display, 
mission  rehearsal  systems,  and  perhaps  the  D/ETB  is  through  the  use 
of  a  common  database  for  imaging  data  and  the  implementation  of 
proven  image  generation  algorithms  in  VHSIC  class  ASICs. 
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Smart  Weapons  Operability  Enhancements  Program 
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ABSTRACT 

Smart  weapon  systems  offer  one  of  the  most  significant  oppor¬ 
tunities  to  offset  threat  numerical'  superiority  with  technology. 
The  Army  has  planned  multi-billion  dollar  investments  in  smart 
munitions  over  the  next  several  years.  A  major  barrier  to  advance¬ 
ment  of  effective  smart  weapons  has  been  the  lack  of  a  systematic 
incorporation  of  the  complexities  of  the  environment  throughout  the 
development  process.  Smart  weapons  must  be  able  to  discriminate 
relatively  small  targets  out  of  large  areas  dominated  by  the  envi¬ 
ronment.  The  utility  of  high  fidelity  target  characterization  is 
compromised  by  the  current  simplistic  description  of  the  environ¬ 
ment.  To  address  this  problem,  the  DOD  is  funding  a  Smart  Weapons 
Operability  Enhancement  (SWOE)  program  as  one  of  its  Balanced 
Technology  Initiatives  (BTI ) .  The  purpose  of  the  SWOE  program  is 
to  provide  the  basis  for  extending  current  smart  weapons  develop¬ 
ment  strategy  to  more  comprehensively  include  consideration  of 
dominant  environmental  conditions  in  expected  areas  of  conflict. 
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The  SWOE  program  is  a  tri-service  effort  managed  by  the  US 
Army  Cold  Regions  Research  and  Engineering  Laboratory  (CRREL) .  The 
objective  of  the  program  is  to  provide  the  hardware  development 
community  with  the  measurement,  modeling,  information  and  simula¬ 
tion  tools  necessary  to  effectively  consider  and  exploit  the 
operational  environment  in  smart  weapons  test  and  evaluation. 
Information  that  defines  the  background  environment  in  terms  sig¬ 
nificantly  relevant  to  proposed  sensor  concepts  will  be  developed 
and  demonstrated  for  two  primary  users:  smart  weapons  system 
"testers/evaluators"  and  "designers".  This  will  provide  the  abili¬ 
ty  to  plan  developmental  test  programs  to  answer  the  most  pressing 
performance  questions,  maximize  the  utility  of  the  test  data  with 
regard  to  performance  in  a  wider  range  of  conditions,  minimize  the 
testing  required  to  complete  proof  of  principal  and  influence  the 
design  of  the  system  as  early  as  possible.  In  the  longer  term, 
SWOE  will  provide  the  ability  to  project  approximate  performance 
prior  to  building  a  brass  board  and  to  forecast  test  results  prior 
to  testing  by  means  of  a  simulation  capability  that  will  impact 
design  in  the  earliest  stages. 

Efforts  were  initiated  during  FY89  to  demonstrate  the  SWOE 
community' s  collective  capabilities  to  model  and  simulate  the 
environment  based  on  first  principles  IR  models  using  real  terrain 
and  measured  data.  This  paper  will  discuss  the  overall  SWOE  pro¬ 
gram  goals  and  objectives  and  present  the  results  of  the  demonstra¬ 
tion  efforts  to  date. 


284 


Presented  at  the 

DIGITAL /ELECTRONIC  TERRAIN  BOARD  SYMPOSIUM 
5-6  October  1989  GACIAC  PR  89-07 


FROM  DATA  BASE  CREATION  TO  PERSPECTIVE  SCENE 
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The  Problem! 

Users  of  mission  planning,  rehearsal  and  simulation  systems  have  been 
plagued  with  problems  which  result  largely  from  overspecialization 
and  incompatibility.  They've  been  faced  with  the  necessity  of  using 
one  system  to  create  the  data  base,  a  second  system  to  process  the 
imagery  and  a  third  system  to  generate  the  perspective  scenes .  To 
further  add  to  the  problem,  some  data  exist  in  pixel  (picture 
element)  form,  other  data  as  polygons  and  still  other  data  not  even 
as  imagery,  at  all.  As  a  result,  a  great  deal  of  their  time  is  spent 
in  learning  and  understanding  these  various  systems  and  data  forms 
rather  than  actually  deriving  useful  benefit  from  the  technology. 

Similarly,  designers  of  each  of  the  systems  mentioned  above  have 
their  tasks  complicated  by  the  apparent  incompatibility  of  the  data 
forms.  They  are  also  usually  unwilling  to  compromise  the  capability 
of  their  technological  tools  in  order  to  accommodate  the  various  data 
forms.  And  with  good  reason:  such  compromises  usually  result  in 
greatly  degraded  performance.  This  is  especially  true  in  perspective 
scene  generation,  where,  for  example,  the  reduction  of  the  number  of 
polygons  that  a  sy - ; . .m  can  generate  immediately  impacts  the  realism 
of  the  scene  and  th«  failure  to  reduce  impacts  the  scene  generation 
time. 

The  Solution: 

Recently,  however,  technological  advances  have  made  it  possible  to 
overcome  these  inherent  problems .  One  approach  involves  the  use  of  a 
"voxel"  data  base.  Voxels  are  volume,  or  three  dimensional,  pixels. 
Such  a  data  base  may  be  built  in  a  pixel  processing  system  and  will 
support  both  polygon  and  voxel  perspective  scene  generatic 

For  example,  a  comprehensive  software  package,  t^  -if ied  by  Image 
Data's  PARA/SERIES  Image  Exploitation  Software,  will  enable  the  user 
to  create  the  data  base,  process  the  pixel  data  and  create 
perspective  scenes,  in  a  single  platform. 

Prior  to  building  the  data  base  it  is  usually  necessary  to  prepare 
the  source  imagery  for  analysis.  This  includes  the  use  of  various 
image  processing  algoritJuns  to  remove  noise,  enhance  the  scene, 
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cartographically  correct  the  image,  define  edges  and  similar 
operations.  At  this  point  the  two  dimensional  image  is  ready  to  be 
incorporated  into  a  three  dimensional  data  base  through  the  addition 
of  the  third  dimension,  elevation. 

Elevation  data  may  be  merged  with  the  source  imagery  or  extracted 
from  it.  Various  techniques  are  available  to  accomplish  this  task. 
Merging  involves  the  use  of  external  elevation  data.  Examples  of 
externally  supplied  elevation  data  sets  are  Digital  Elevation  Models 
(DEM)  from  the  US  Geological  Survey  and  Digital  Terrain  Elevation 
Data  ( DTED )  from  the  Defense  Mapping  Agency.  These  are  usually 
provided  in  non- image  form  and  must  first  be  converted.  In  doing  so, 
the  elevation  postings  must  be  arranged  in  a  raster  format  similar  to 
the  image.  Then,  an  intensity  assignment  is  made  at  each  location, 
proportional  to  the  elevation  value  of  the  original  posting.  This 
then  forms  a  black  and  white  image,  representative  of  the  elevation, 
where  lighter  shades  of  gray  indicate  higher  elevations.  If  the 
postings  are  more  coarsely  distributed  than  the  pixels  in  the  source 
image,  an  interpolative  process  may  be  employed  to  establish  a  unique 
elevation  value  for  each  pixel  location.  Conversely,  if  the  postings 
are  denser  than  the  pixel  distribution,  averaging  may  be  employed. 
Since  the  image  and  elevation  data  were  separately  collected,  their 
formats  will  not  be  the  same,  geometrically.  Therefore,  an 
additional  step  is  needed  when  using  external  elevation  sources. 
This  involves  geometric  transformation  of  one  image  to  the  other, 
typically  through  tiepoint  selection  and  polynomial  warping. 

An  alternative  means  of  incorporating  elevation  information  into  the 
data  base  is  through  extraction.  The  extraction  of  elevation  data  is 
usually  accomplished  through  one  of  two  methods,  depending  on  the 
source  imagery  available.  If  the  image  is  monoscopic  (i.e.  a  single 
image)  shadow  length  and  solar  elevation  angle  may  be  used  to 
calculate  the  elevation  of  features  within  the  scene.  This  process 
is  usually  very  interactive  although  morphological  algorithms  may  be 
used  in  conjunction  with  intensity  thresholds  to  define  shadow  edges 
in  a  more  automated  fashion.  Obviously,  this  process  may  be  employed 
only  for  those  images  where  shadows,  related  to  elevation,  actually 
exist.  This  eliminates  certain  sensors  operating  in  wavelength 
bandwidths  where  shadows  are  not  relevant  to  elevation.  Also,  it 
does  not  work  for  those  scenes  where  shadows  are  not  present,  such  as 
at  night  in  the  visible  spectrum.  Finally,  shadow  length  can  be  very 
complex  and  difficult  to  measure,  such  as  in  cases  where  one  shadow 
falls  upon  others.  However,  there  are  times  where  this  is  the  only 
means  available  for  obtaining  any  elevation  information  about  a 
geographic  area. 

A  much  more  automated  potential  exists  where  stereoscopic  imagery  is 
available.  Stereo  sets  (pairs  or  greater)  may  be  used  to  provide 
elevation  information  if  the  altitude  and  the  convergence  angle  of 
the  sensors  are  known.  It  is  important  to  note  that  the  stereo 


imagery  could  be  collected  by  a  single  sensor  providing  multiple 
images  displaced  in  time.  It  is  also  possible  to  extract  elevation 
from  stereo  sets  where  the  sensors  may  not  be  temporally  displaced 
but  on  different  platforms,  such  as  satellites  and  aircraft. 
Further,  the  sensors  may  even  generate  data  in  completely  different 
portions  of  the  spectrum,  sur  h  as  thermal  and  radar  imagery.  And 
there  may  exist  different  combinations  of  these  variables  with  which 
the  data  base  creator  must  work.  However,  given  the  availability  of 
stereo  imagery  and  the  use  of  edge  and/or  gray  level  correlation 
along  with  the  appropriate  geometric  transformation  algorithms, 
elevation  data  may  be  obtained  with  a  high  degree  of  accuracy  in  an 
automated  fashion. 

Neither  of  the  extraction  approaches  covered  above  requires  geometric 
transformation  to  superimpose  the  image  data  with  the  elevation 
data.  This  is  true  since  the  elevation  data  was  derived  directly 
from  the  source  imagery.  Similarly,  no  interpolation  will  be 
required  since  the  elevation  data  can  be  extracted  for  every  pixel 
location  in  the  image. 

At  this  point  other  feature  attributes  may  be  determined  to  create  a 
more  comprehensive  data  base.  Using  techniques  such  as  supervised 
and  unsupervised  Classification  of  multi-spectral  imagery  or  thermal 
classification  of  infrared  data  will  permit  spectral  attributes  to  be 
assigned  to  each  voxel  in  the  data  base.  Likewise,  textural  and 
shape  attributes  may  be  derived  and  assigned.  This  then  creates  a 
surface  material  classification  which  may  be  included  in  perspective 
scenes  for  additional  functionality.  Knowledge  of  surface  material 
classes  is  also  extremely  useful  when  the  user  wishes  to  simulate  the 
response  of  sensors  elsewhere  in  the  electromagnetic  spectrum.  If 
the  spectral  signature  of  the  feature  is  known,  it  is  feasible  to 
convert  its  radiometric  response  to  another  portion  of  the  spectrum. 
As  an  example,  if  a  metal  structure  is  identified  in  a  visible  image, 
one  can  simulate  its  thermal  characteristics  under  differing  solar 
illumination  conditions. 

Why  Parallel? 

Up  to  this  point,  only  a  limited  advantage  could  be  obtained  from  a 
parallel  processing  platform.  Algorithms  such  as  Fourier  transforms, 
multispectral  classifications  and  geometric  manipulations  derived 
benefit  from  parallel  implementation. 

However,  it  is  in  the  area  of  perspective  scene  generation  that 
dramatic  improvement  in  performance  can  be  achieved  through 
parallelism.  The  data  base  which  has  been  created  can  be  used  to 
generate  photorealistic  perspective  scenes  for  mission  planning, 
mission  rehearsal  and  simulation  applications.  In  order  to  generate 
these  scenes  rapidly  and  still  preserve  all  of  the  resolution 
inherent  in  the  data  base,  an  array  of  high  performance  parallel 
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processors  with  closely  coupled,  shared  memory  becomes 
indispensable.  An  architecture  of  this  type  running  a  constant-time, 
ray  trace  algorithm  will  enable  the  user  to  achieve  real-time  scene 
generation  (i.e.  30/60  fps.)  Further,  when  this  parallel 
architecture  is  married  to  a  polygon  generator  and  Z-buffer  memory, 
dynamic  objects  may  be  inserted  into  the  scene  under  independent 
control  and  a  truly  photorealistic  simulation  capability  results. 
Further  realism  may  be  added  to  the  overall  system  through  the 
addition  of  shading  and  lighting  algorithms  to  the  software  library. 
That,  however,  is  the  topic  of  another  discussion. 

The  Conclusion: 

Recent  developments  in  parallel  architectures,  commodity  processors 
and  advanced  algorithms  eliminate  the  need  for  multiple  systems  and 
data  bases  in  applications  involving  perspective  scene  generation. 
Mission  planners  may  now  use  a  single  system  to  accomplish  their 
goals.  Further,  this  approach  has  the  collateral  benefit  of  device 
independence.  Users  have  the  opportunity  to  avail  themselves  of 
current  hardware  technology  with  a  minimum  amount  of  cost  and  delay 
when  transporting  their  applications  software  systems  to  these  newer 
and  more  powerful  platforms.  Efficiency  is  greatly  improved  since 
users  continue  to  work  with  familiar  software  while  benefiting  from 
technology  advancements . 

The  following  series  of  images  were  obtained  from  a  single  parallel 
platform  utilizing  the  PARA/SERIES  Software  System  developed  by  Image 
Data  Corporation.  They  clearly  indicate  that  it  is  possible  to 
process  imagery,  create  a  3-D  data  base  and  generate  photorealistic 
perspective  scenes  with  a  common  platform  and  software. 
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The  photo-composite  above  shows  an  original  LANDS  AT  image  of  the  San  Gabriel  Valley 
in  the  upper  left  hand  quadrant.  To  the  right  of  it  is  a  binary  edge  image  for  extracting  shape 
information.  Below  that  is  the  result  of  an  unsupervised  multispectral  classification  showing 
22  classes.  The  final  quadrant  is  one  of  a  sequence  of  perspective  scenes  of  the  area.  The 
“fly  through”  was  accomplished  at  near-real-time  rates. 


The  photo-composite  above  shows  two  techniques  for  building 
an  elevation  data  base.  The  upper  pair  depicts  an  original 
Landsat  3-band  multispectral  image  alongside  a  US  Geological 
Survey  elevation  data  set  converted  into  a  gray  scale  image 
and  geometrically  warped  to  superimpose  on  the  Landsat 
scene.  The  lower  pair  consists  of  an  aerial  photo  of  the 
CalTech  campus  alongside  a  partial  elevation  extraction  of 
several  of  the  buildings .  The  technique  used  here  measures 
the  length  of  shadows  at  a  known  solar  elevation  ang i 
should  be  noted  that  many  roofs  exhibit  varying  in  can-  .w..  , 

indicating  complex  structures  such  as  air  conditioning  units 
and  dormers,  casting  independent  shadows. 
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The  photo-compos xte  above  compares  what  may  be  called  2.5-D 
perspective  scene  renditions  to  full  3-D  renditions.  The 
upper  left  quadrant  is  again  the  aerial  photo  of  the  CalTech 
campus.  The  upper  right  quadrant  shows  a  perspective  scene 
in  which  the  detail  of  the  sides  of  the  buildings  was 
unavailable.  The  lower  right  incorporates  that  missing  data 
through  the  addition  of  ground-level  photography  merged  into 
the  data  base.  This  quadrant  actually  depicts  a  perspective 
view  of  the  building  with  the  observer's  viewpoint  located 
at  ground  level.  The  lower  left  quadrant  further  indicates 
the  nature  of  a  true  3-D  data  base.  Here,  the  observer  is 
placed  inside  one  of  the  rooms  within  the  building 
able  to  look  out  the  window  at  the  outside  scene, 
though  no  data  relative  to  the  inside  of  the  building  was  in 
the  data  base. 
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THE  TACTICAL  WEAPON  GUIDANCE  AND  CONTROL 
INFORMATION  ANALYSIS  CENTER  (GACIAC) 


CACIAC  is  a  DoD  Information  Analysis  Center  operated  by  IIT  Research  Institute  under 
the  technical  sponsorship  of  the  Joint  Service  Cuidance  and  Control  Committee  with 
members  for  OUSDRE,  Army,  Navy,  Air  Force,  and  DARPA.  The  U.S.  Army  Missile 
Command  provides  the  Contracting  Officer's  Technical  Representative.  Its  mission  is  to 
assist  the  tactical  weapon  guidance  and  control  community  by  encouraging  and  facilitat¬ 
ing  the  exchange  and  dissemination  of  technical  data  and  information  for  the  purpose  of 
effecting  coordination  of  research,  exploratory  development,  and  advanced  technology 
demonstrations.  To  accomplish  this,  GACIAC's  functions  are  to: 

1 .  Develop  a  machine-readable  bibliographic  data  base  - 
currently  containing  over  36,000  entries; 

2.  Collect,  review,  and  store  pertinent  documents  in  its  field  of  interest  - 
the  library  contains  over  1 1,000  reports; 

3.  Analyze,  appraise  and  summarize  information  and  data  on  selected 
subjects; 

4.  Disseminate  information  through  the  CACIAC  Bulletin,  bibliographies, 
state-of-art  summaries,  technology  assessments,  handbooks,  special 
reports,  and  conferences; 

5.  Respond  to  technical  inquiries  related  to  tactical  weapon  guidance 
and  control;  and 

6.  Provide  technical  and  administrative  support  to  the  Joint  Service 
Guidance  and  Control  Committee  (JSGCC). 

The  products  and  services  of  GACIAC  are  available  to  qualified  industrial  users  through 
a  subscription  plan  or  individual  sales.  Government  personnel  are  eligible  for  products 
and  services  under  block  funding  provided  by  the  Army,  Navy,  Air  Force  and  DARPA.  A 
written  request  on  government  stationery  is  required  to  receive  all  the  products  as  a 
government  subscriber. 

Further  information  regarding  GACIAC  services,  products,  participation  plan,  or  addi¬ 
tional  copies  of  these  Proceedings  may  be  obtained  by  writing  or  calling:  GACIAC,  IIT 
Research  Institute,  10  West  35th  Street,  Chicago,  Illinois  60616-3799,  Area  Code  312, 
567-4519  or  567-4526. 


